-- Logs begin at Wed 2022-08-24 09:11:08 UTC, end at Wed 2022-08-24 10:32:08 UTC. -- Aug 24 09:45:30.543442 dv-cinder-854269-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 09:45:30.553640 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 09:45:30 2022] *** Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: nodename: dv-cinder-854269-2-iscsi Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: machine: x86_64 Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: clock source: unix Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: pcre jit disabled Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: detected number of CPU cores: 4 Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: current working directory: / Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: detected binary path: /usr/bin/uwsgi-core Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: your processes number limit is 47680 Aug 24 09:45:30.555677 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: your memory page size is 4096 bytes Aug 24 09:45:30.556404 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: detected max file descriptor number: 2048 Aug 24 09:45:30.556404 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: lock engine: pthread robust mutexes Aug 24 09:45:30.556404 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: thunder lock: enabled Aug 24 09:45:30.556404 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 09:45:30.556404 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 09:45:30.595519 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: Python main interpreter initialized at 0x555e6b8129c0 Aug 24 09:45:30.595519 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: python threads support enabled Aug 24 09:45:30.595519 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: your server socket listen backlog is limited to 100 connections Aug 24 09:45:30.595519 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: your mercy for graceful operations on workers is 90 seconds Aug 24 09:45:30.595899 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 09:45:30.595978 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: *** Operational MODE: preforking *** Aug 24 09:45:30.596218 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: *** uWSGI is running in multiple interpreter mode *** Aug 24 09:45:30.596218 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: spawned uWSGI master process (pid: 81455) Aug 24 09:45:30.596414 dv-cinder-854269-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 09:45:30.597458 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: spawned uWSGI worker 1 (pid: 81466, cores: 1) Aug 24 09:45:30.597458 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: spawned uWSGI worker 2 (pid: 81467, cores: 1) Aug 24 09:45:30.597458 dv-cinder-854269-2-iscsi devstack@keystone.service[81455]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 09:45:34.953505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:45:34.953505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: from cryptography.utils import int_from_bytes Aug 24 09:45:34.954300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:45:34.954300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: from cryptography.utils import int_from_bytes Aug 24 09:45:34.954421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:45:34.954421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: from cryptography.utils import int_from_bytes Aug 24 09:45:34.954421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 09:45:34.954421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: from cryptography.utils import int_from_bytes Aug 24 09:45:35.005103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:45:35.005277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:45:35.005360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 09:45:35.005474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] command line args: [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:45:35.005553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 09:45:35.005669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] command line args: [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 09:45:35.005758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:45:35.005912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ================================================================================ {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:45:35.005990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 09:45:35.006108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ================================================================================ {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 09:45:35.006186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] admin_token = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] admin_token = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] config_dir = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] config_dir = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] config_source = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.006852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.007057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] config_source = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.007703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] debug = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.007838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] debug = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.007918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.008940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_config_append = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_config_append = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_dir = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_dir = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.009941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_file = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_file = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.010926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.011911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.012976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.78/identity {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.013927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] publish_errors = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] publish_errors = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.014910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] standard_threads = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] standard_threads = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.015968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] use_journal = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] use_journal = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] use_json = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] use_json = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] use_stderr = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] use_stderr = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] use_syslog = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] use_syslog = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.016949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 09:45:35.017130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.017949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.018916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.019925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.020892 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.021888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.022922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.023882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.024997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.025979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.026990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.027936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.028896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029614 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.029997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.030947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.031910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.032956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.033933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.034903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.035922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.036844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.037983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.038963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.039915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.040942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.041955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.external = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.external = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.042984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.password = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.password = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] auth.token = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] auth.token = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.043940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.044847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.045990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.046960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.047990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.048868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.049850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.050978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.051990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.052972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.053680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.054279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.054445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.054524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.054691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.054771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.055856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.056848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.057867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058802 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.058960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.059948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.060935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.061975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.062976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.063987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.064974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.065931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.066929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.067910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.068951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.069122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.069200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.069387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.070963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.071946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.072917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.073880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.074997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.075991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.076968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] role.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.077919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] role.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] role.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] role.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.078967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.079989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.080970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.081773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.082829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.083854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.084942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.085949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.086909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.087917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.088900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.089877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.090902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.connection = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.connection = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.091962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.092912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.093879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.094854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81467) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:45:35.095482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 09:45:35.095914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81466) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 09:45:35.125220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.125453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.125539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.125681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.125765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.125922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.135085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.135356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.163596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.163852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.164265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.164502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.219084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.219230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.268709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.268943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.269033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.269199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.430736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.430960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.431942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.432082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.432302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.456992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.457772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.469828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.469951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.482709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.482988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.483932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.484014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.484148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.484233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81466) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.484367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81467) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 09:45:35.501068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.501227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.501332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:45:35.501439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:45:35.501613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.501784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.501870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:45:35.501985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 09:45:35.502077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.502262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.502342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.502529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.502852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.503039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.503120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.503308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.513332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.513618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.513701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.513897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.513981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.514189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.514272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.514497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.521196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.521482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.524998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.525294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.525375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.525582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.525663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.525868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.525948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.526972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.527952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.528036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.528229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.551518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.551619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.553014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.553113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.553305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.553412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.558707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.558800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.559001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.559083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.559290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.559371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.565661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.565760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.565955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.566036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.570238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.570329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.570535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.570629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.570824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.570904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.571979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.572073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.572251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.572332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.589504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.589606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.589787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.589883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.593678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.593762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.596909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.596993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.597220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.597303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.601978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.602142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.605149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.605348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.608414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.608625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.608707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.608936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.609904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.610131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.626166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.626473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.626749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.627020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.628932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.629253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.629533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.629809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.634965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.636517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.636800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.637340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.638877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.639164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.639458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.639759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.640087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.640390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.640675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.641859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.642163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.642468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.642773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.643070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.652594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.652880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.653165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.653448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.653733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.654016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.654307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.655145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.655441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.655730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.656044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.656332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.656617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.656903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.669747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.672492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.672833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.675560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.675721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.677193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.678540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.679071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.679071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.679263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.680195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.681692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.681985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.682292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.685753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.687212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.688823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.690118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.690297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.690413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.690711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.691000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.693288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.693591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.693894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.694766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.699476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.699774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.700088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.702740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.703040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.703331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.707037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81467) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.707334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.707621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.707939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.708224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.708505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.708796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.709081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.709365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.709707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.710681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.710781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.710857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81466) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 09:45:35.710993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.711073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.711261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81467) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.711341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.711627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.711945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.712230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.712524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.712806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.713095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.713383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.713662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.713940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.714223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.714502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81466) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 09:45:35.745607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555e6b8129c0 pid: 81467 (default app) Aug 24 09:45:35.748975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x555e6b8129c0 pid: 81466 (default app) Aug 24 09:45:35.749067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344b81-8d5e-4244-b6a7-5c1572aeba23 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:35.749294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344b81-8d5e-4244-b6a7-5c1572aeba23 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:35.749568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32344b81-8d5e-4244-b6a7-5c1572aeba23 None None] PATH_INFO: `/v3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:35.754012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1/1] 10.222.0.78 () {50 vars in 791 bytes} [Wed Aug 24 09:45:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:42.336244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:42.336849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:42.336849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:42.341090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1/2] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:45:42.348190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:42.348610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:42.348738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:42.419158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: INFO dbcounter [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] Registered counter for database keystone Aug 24 09:45:42.438745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG oslo_db.sqlalchemy.engines [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81467) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:45:42.445112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writer thread running {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:45:42.623662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG passlib.handlers.bcrypt [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81467) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:45:42.624036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG passlib.handlers.bcrypt [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81467) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:45:42.678897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:42.681370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:42.817490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 2/3] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 09:45:43.262673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:43.331135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: INFO dbcounter [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] Registered counter for database keystone Aug 24 09:45:43.352496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG oslo_db.sqlalchemy.engines [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81466) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 09:45:43.358595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writer thread running {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 09:45:43.479503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad4b3a-e3df-45a2-a20f-1ca2b6e25e51 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:43.483763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:43.484413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:43.484693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:43.484967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:43.493586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:43.546432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:45:43.546728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c0667-bef1-4d41-910b-985023849c2c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:43.556464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 2/4] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Aug 24 09:45:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 09:45:43.562334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:43.576994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f31a11-ed6d-423c-9274-51f80680244f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:43.582229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:43.583544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:43.583852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:43.584129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] PATH_INFO: `/v3/projects/7790bbe18b09475b81076e89944d2568` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:43.585674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] RBAC: Authorizing `identity:get_project(project_id=7790bbe18b09475b81076e89944d2568)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:43.639960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 09:45:43.640303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc7a81-6f6e-40cc-a93c-378c109117fb admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:43.645823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 3/5] 10.222.0.78 () {56 vars in 1207 bytes} [Wed Aug 24 09:45:43 2022] GET /identity/v3/projects/7790bbe18b09475b81076e89944d2568 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:45.520595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1bf-9e19-48c7-b744-fdcec8d61c67 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:45.521065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1bf-9e19-48c7-b744-fdcec8d61c67 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:45.521156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f1bf-9e19-48c7-b744-fdcec8d61c67 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:45.525240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 3/6] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:45.530703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:45.531074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:45.531252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:45.552621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:45.554959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:45.562766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 4/7] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 09:45:45.828245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f1bf-9e19-48c7-b744-fdcec8d61c67 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:45.841019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83f1bf-9e19-48c7-b744-fdcec8d61c67 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:45.845176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:45.846384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:45.846765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:45.847035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:45.854539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:45.856338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb8077-00fc-47dd-bfb2-14afe6196717 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:45.865090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 4/8] 10.222.0.78 () {58 vars in 1324 bytes} [Wed Aug 24 09:45:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:45:45.870490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:45.880811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f65b5-5369-4a70-a129-bb4ee03e10bb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:45.885140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:45.885795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:45.886078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:45.886351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:45.887663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:45.889523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3dd92-9071-4d3e-b6a6-c5d0c1caf86b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:45.891048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 5/9] 10.222.0.78 () {56 vars in 1198 bytes} [Wed Aug 24 09:45:45 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:47.854023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28527b4c-e65f-4556-9501-078de7b12d77 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:47.854533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28527b4c-e65f-4556-9501-078de7b12d77 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:47.854533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28527b4c-e65f-4556-9501-078de7b12d77 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:47.855859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 5/10] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:47.861867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:47.862295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:47.862427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:47.881761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:47.882962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:47.910960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324d02-a740-4e56-bb30-2e9dcee263fb None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:47.911245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324d02-a740-4e56-bb30-2e9dcee263fb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:47.911522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324d02-a740-4e56-bb30-2e9dcee263fb None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:47.912898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 6/11] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:45:47.917911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:47.918198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:47.918481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:47.960298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG passlib.handlers.bcrypt [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81466) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 09:45:47.960720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG passlib.handlers.bcrypt [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81466) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 09:45:47.963912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 6/12] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:48.007124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:48.008749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:48.107466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 7/13] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:45:48.232915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.257261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.337500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4cc59-58f5-4149-9305-b0c123809f07 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.370777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.371349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.371628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.371930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.376523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.383489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:45:48.392320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:45:48.393269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:45:48.394372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 7/14] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:45:48.399096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.483687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df437a-ac71-4be5-b067-188e4e216d88 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.483884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.488166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.505282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e287094-9c59-4e34-a921-2a9617b5d80e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.524842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.525712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.525998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.526276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.539267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.539906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.540192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.540468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.541145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:48.541449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.542855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:48.555044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 8/15] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:48.560575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.593870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.596236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:48.658958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa22b50-09d9-4d38-8b9b-bf8a516ea900 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.692201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.692829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.693108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.693378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.694512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.694940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 8/16] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:45:48.695996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:48.697658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 9/17] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:48.700537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.702927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.703322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.718893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.800056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b76dc5-6d66-4bd1-9fa7-97ac3bba1bd6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.806698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f0b0b-4f0a-4060-bb96-68ec8474b38a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.833233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.834084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.834369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.834643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.839576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.840242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.840504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.840771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.841874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.843316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:48.845455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 10/18] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:48.851094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.854289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.855770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:45:48.874204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:45:48.875268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:45:48.876801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:45:48.876801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:45:48.876801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:45:48.876801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 9/19] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:45:48.881854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:48.935383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.941810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:48.955668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a400a-0a18-4eb1-8d41-1b750ef5a9ac None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.986210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1da9c-76f3-47f4-9efc-f0d01b0b1ebd None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:48.989842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:48.990455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:48.990739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:48.991014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:48.991742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.domain.id=default, user.id=0f68be8cf7b24b869bf665ebc09aa5e5) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:48.992081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:48.993550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fdb3e-b701-4037-8698-8301d1a2fcc6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:49.015814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 11/20] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/role_assignments?user.id=0f68be8cf7b24b869bf665ebc09aa5e5&scope.domain.id=default&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:49.021789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:49.022427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:49.022710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:49.022985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:49.023651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:49.023986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:49.025442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8dff3-12a6-4592-a278-58c53d88ff58 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:49.046394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 10/21] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:51.077672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77bdb7-8034-4814-a8e1-992c1e4d349b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.078707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77bdb7-8034-4814-a8e1-992c1e4d349b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.079146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77bdb7-8034-4814-a8e1-992c1e4d349b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.080263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 12/22] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:51.085623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593998-cbb7-4d86-bf3e-9f71b64b64f6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.086419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593998-cbb7-4d86-bf3e-9f71b64b64f6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.086901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b593998-cbb7-4d86-bf3e-9f71b64b64f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.109075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8b593998-cbb7-4d86-bf3e-9f71b64b64f6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:51.110616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8b593998-cbb7-4d86-bf3e-9f71b64b64f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:51.202198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 11/23] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:51.604152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45386192-7d85-4d3c-9a3d-1fa702d4ebf2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.604850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45386192-7d85-4d3c-9a3d-1fa702d4ebf2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.605371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45386192-7d85-4d3c-9a3d-1fa702d4ebf2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.606910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 13/24] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:51.613315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca954-9e35-4cff-9f91-59c9ebb0b365 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.613814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca954-9e35-4cff-9f91-59c9ebb0b365 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.614276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ca954-9e35-4cff-9f91-59c9ebb0b365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.639424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-747ca954-9e35-4cff-9f91-59c9ebb0b365 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:51.641169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-747ca954-9e35-4cff-9f91-59c9ebb0b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:51.692520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45386192-7d85-4d3c-9a3d-1fa702d4ebf2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:51.697573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:51.741553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 12/25] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:45:51.754456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ccc4-9a0b-41c3-86dc-778f05e3963a None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.754749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ccc4-9a0b-41c3-86dc-778f05e3963a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.755029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452ccc4-9a0b-41c3-86dc-778f05e3963a None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.756376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 13/26] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:45:51.773727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7f04d-5aa5-4766-8349-0cf884654c28 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.774260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7f04d-5aa5-4766-8349-0cf884654c28 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.774742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7f04d-5aa5-4766-8349-0cf884654c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.808802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ade7f04d-5aa5-4766-8349-0cf884654c28 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:51.816184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ade7f04d-5aa5-4766-8349-0cf884654c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:51.893757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45386192-7d85-4d3c-9a3d-1fa702d4ebf2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:51.931557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 14/27] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:45:51.933089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7e19-9463-4ffe-8508-cdaa98b05f9c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.933623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7e19-9463-4ffe-8508-cdaa98b05f9c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.935671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b7e19-9463-4ffe-8508-cdaa98b05f9c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.937674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 15/28] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:51.943449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.943952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.944383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.965476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:51.966370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:51.966793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:51.967220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:51.980125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:51.982390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:51.987954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:45:51.993469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:45:51.994495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:51.995463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:45:52.003708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 14/29] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:45:52.012769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.022884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:52.120280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 16/30] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:52.129013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.195300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:52.213682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38aaf6-891b-4014-ba2a-d90fc0c23d75 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.270092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49188a84-c1a7-444f-ac0f-14ae48e15e03 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.300657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.301527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.301953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.302372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.303264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:52.303828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.305760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.306600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.307232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.307565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.307964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.322748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 15/31] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:52.329147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.332735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.334447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:45:52.351587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:45:52.352613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:45:52.353409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:45:52.353409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:45:52.353409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:45:52.355387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 17/32] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:45:52.361601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.469670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:52.500279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:52.513683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757eeb6e-3f40-482c-8ccf-1c1b8a3d4368 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.526481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5e7ac-c6a1-48c3-9174-043b520e3ea2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.551549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.552444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.552884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.553300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.554088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:52.554533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.556122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.569388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 18/33] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:52.570451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.574296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.575943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.576229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.576506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.577951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.579421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.581288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 16/34] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:45:52 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:52.582737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.697708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de991f-8876-4a50-95d5-20d52e75a8ca None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.701250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615125c-8493-44b1-bffa-c7aedb8ebb0c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:52.738168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.738890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.739253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.739531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.739753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:52.740233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.740388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:52.740851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:52.741278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:52.741710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.741942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:52.743320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:52.747400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:52.867036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 19/35] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 24 09:45:52 2022] POST /identity/v3/roles => generated 219 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:45:52.867356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 17/36] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 24 09:45:52 2022] POST /identity/v3/roles => generated 225 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:45:52.876106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.876185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:52.904168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:53.097808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe0c8b-3e36-4535-b3d1-fe6826fd5fb2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:53.102572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3ec08-3245-430f-9c87-952cbaa017e9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:53.142901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:53.144350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.144863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.145287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.146359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:53.147988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959cf11-6c70-48ae-9374-a1074aa928e3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:53.148115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:53.148689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.148978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.149261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.150486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:53.152073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab586a4f-a1fd-46f5-96cb-1beb3ac9eec0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:53.153743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 18/37] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:53.155008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd436-f7bf-4f3c-b32f-8eebadcd60f8 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.156011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd436-f7bf-4f3c-b32f-8eebadcd60f8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.156553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd436-f7bf-4f3c-b32f-8eebadcd60f8 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.158298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 19/38] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:53.159579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e19f3-9d8d-43e4-a0bc-cceeeaf187b2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.160290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e19f3-9d8d-43e4-a0bc-cceeeaf187b2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.160725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475e19f3-9d8d-43e4-a0bc-cceeeaf187b2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.161956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 20/39] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:53.164604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e19f3-9d8d-43e4-a0bc-cceeeaf187b2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:53.180243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 20/40] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:45:52 2022] POST /identity/v3/projects => generated 301 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:45:53.181410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0084a-9f16-4e9e-b8b0-bf8c272f7de6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.181707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0084a-9f16-4e9e-b8b0-bf8c272f7de6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.181993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb0084a-9f16-4e9e-b8b0-bf8c272f7de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.207268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bcb0084a-9f16-4e9e-b8b0-bf8c272f7de6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:53.209086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bcb0084a-9f16-4e9e-b8b0-bf8c272f7de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:53.240173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:53.277686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:53.291248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475e19f3-9d8d-43e4-a0bc-cceeeaf187b2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:53.291939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 21/41] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:53.293154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.293443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.293719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.313341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:53.314793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:53.339290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:53.340107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.340466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.340817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] PATH_INFO: `/v3/domains/default/users/0f68be8cf7b24b869bf665ebc09aa5e5/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.343625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0f68be8cf7b24b869bf665ebc09aa5e5, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:53.345126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:53.364504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 21/42] 10.222.0.78 () {58 vars in 1361 bytes} [Wed Aug 24 09:45:53 2022] PUT /identity/v3/domains/default/users/0f68be8cf7b24b869bf665ebc09aa5e5/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:45:53.408729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 22/43] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:53.578411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:53.682082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea7292-1ae4-407b-b67e-f487b0cf51d5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:53.693641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:53.709931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:53.725631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:53.726526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.726913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.727197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.727951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:53.729405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:53.745991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 22/44] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 24 09:45:53 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:45:53.816124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52f944-2001-44c6-b74f-d02ec679c723 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:53.839177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:53.855764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:53.856448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:53.856734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:53.857011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:53.858840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:53.860394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33f5cb-39b5-4b04-b82e-55b1d89d18d1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:53.862083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 23/45] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:53.867906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:53.979991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eda9b-0974-465c-a5ab-093a0b4b20d9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:54.013637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:54.014921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:54.015250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:54.015532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:54.016203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:54.017661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ada488-7a8c-4e85-b392-cc3803fd8157 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:54.158548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 23/46] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:45:53 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:45:55.306172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc98b-86df-4fb1-a320-f00a159d1fd3 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:55.306695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc98b-86df-4fb1-a320-f00a159d1fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:55.306777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc98b-86df-4fb1-a320-f00a159d1fd3 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:55.308185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 24/47] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:55.313699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:55.314097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:55.314384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:55.332977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:55.334509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:55.361271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:55.402088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 24/48] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:55.680278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc98b-86df-4fb1-a320-f00a159d1fd3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:55.783630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc98b-86df-4fb1-a320-f00a159d1fd3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:55.818437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:55.819067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:55.819350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:55.819625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:55.831748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:45:55.837989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:45:55.838932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:45:55.839836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 25/49] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:45:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:45:55.845119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:55.943293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba9d99-2f90-4aac-8f37-12d3dc2c6029 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:55.976363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:55.976973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:55.977257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:55.977531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:55.978199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:55.978507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:55.979957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb280eba-2dd4-4b77-bf51-12707246220a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:55.988638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 25/50] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:45:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:55.994247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:56.043574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:56.053571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de992-9d29-4c4a-adf3-c903ec5832a5 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.053867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de992-9d29-4c4a-adf3-c903ec5832a5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.054153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0de992-9d29-4c4a-adf3-c903ec5832a5 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.055509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 26/51] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:45:56.061600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.061884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.062163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.079451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:56.080684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:56.096401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04244c0-9321-4fee-a587-2e8357eb8f59 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:56.101018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:56.131235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:56.131910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.132198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.132484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.133863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:56.135327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:56.136965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 26/52] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:45:55 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:56.142974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:56.149829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 27/53] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:56.245616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d31758e-a2a6-4cd0-a8f2-28f227fb749d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:56.261718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:56.280533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:56.281167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.281451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.281731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.282865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:56.284370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:56.285957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 27/54] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:56.291734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:56.389868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d8fae-8c95-4490-8895-0d112d6bed56 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:56.416327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:56.422731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:56.423352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.423636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.423943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.424687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.domain.id=default, user.id=0f68be8cf7b24b869bf665ebc09aa5e5) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:45:56.425003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:56.426489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:56.433094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 28/55] 10.222.0.78 () {56 vars in 1322 bytes} [Wed Aug 24 09:45:56 2022] GET /identity/v3/role_assignments?user.id=0f68be8cf7b24b869bf665ebc09aa5e5&scope.domain.id=default&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:56.629443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e9975-1d14-4fa0-a459-9a229c94a7d8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:56.666267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:56.666906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.667193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.667471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.668639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:56.670840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35188575-fcc2-4d48-9c0d-4e9b618f5a2a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:56.672482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 28/56] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:45:56.678082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:56.699536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:56.775760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27230f43-6ba8-4326-93bd-b465c15a824c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:56.808943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:56.809551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:56.809837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:56.810144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:56.811017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:56.812515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6425382d-703e-4b45-bf71-86dede1fb80a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:56.838721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 29/57] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Aug 24 09:45:56 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:45:58.734004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f640f-128d-4eb0-b77f-eedeec025870 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:58.734513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f640f-128d-4eb0-b77f-eedeec025870 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:58.734513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f640f-128d-4eb0-b77f-eedeec025870 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:58.735871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 29/58] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:45:58.741051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:58.741448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:58.741610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:58.760292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:45:58.761503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:45:58.827808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 30/59] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:45:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:45:59.100437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f640f-128d-4eb0-b77f-eedeec025870 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:59.154388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:59.201429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f640f-128d-4eb0-b77f-eedeec025870 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:59.234314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:59.234935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:59.235218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:59.235492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:59.236635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:59.238075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabac99d-7c2c-482e-aa73-0c2526013d45 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:59.239638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 30/60] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:45:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:45:59.245477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:45:59.248791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:45:59.345262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef749f5-8488-43e4-91a8-8bf78d56a936 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:45:59.378834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:45:59.379386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:45:59.379673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:45:59.379981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:45:59.380639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:45:59.382105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d552e-7b9e-4194-9a11-a772fff6c3d6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:45:59.409141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 31/61] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:45:59 2022] POST /identity/v3/users => generated 275 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:02.367877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607f21-6b26-47bb-836a-a87f0d5b5065 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.369149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607f21-6b26-47bb-836a-a87f0d5b5065 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.369640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe607f21-6b26-47bb-836a-a87f0d5b5065 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.372131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 31/62] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:02.378766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb0847-6003-418b-84c6-921054ef7d5e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.379770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb0847-6003-418b-84c6-921054ef7d5e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.380651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb0847-6003-418b-84c6-921054ef7d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.403388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-fcfb0847-6003-418b-84c6-921054ef7d5e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:02.404965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fcfb0847-6003-418b-84c6-921054ef7d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:02.462514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:02.486151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 32/63] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:02.592564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c55a6-5c7e-439d-8069-6f1ecd2d71d8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.593194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c55a6-5c7e-439d-8069-6f1ecd2d71d8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.593694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c55a6-5c7e-439d-8069-6f1ecd2d71d8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.595277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 32/64] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:02.600809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.601356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.601832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.628498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:02.629617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:02.719818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2039c-0139-4093-a132-e018de8e48d8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.719818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2039c-0139-4093-a132-e018de8e48d8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.719818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2039c-0139-4093-a132-e018de8e48d8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.720873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 33/65] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:02.726758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 33/66] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:02.727389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.727998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.728498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.760017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:02.762115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:02.842866 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 34/67] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:02.845437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:02.949975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e52595e-fb7e-4e8d-949f-9f088099a10a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:02.968774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:02.986281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:02.986937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:02.987438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:02.987722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:02.999745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:03.004653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:03.005584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:03.006894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 34/68] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:03.012482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.035102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.109161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.173075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b48b48-173a-4a8b-9ab5-f43576c79ac8 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.177702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98365372-13f2-4c56-8983-192325a3a4af None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.212545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.213157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.213366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.213644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.214880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:03.215178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.216781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93abcd3-90c8-424d-ac26-19b4763eab59 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.220835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.221477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.221764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.222041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.228965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 35/69] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:03.230962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc1b1-fd53-4c64-b29c-992bdece682c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.231253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc1b1-fd53-4c64-b29c-992bdece682c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.231532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363bc1b1-fd53-4c64-b29c-992bdece682c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.232915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 36/70] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:03.233037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:03.237478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc1b1-fd53-4c64-b29c-992bdece682c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-b0303a0b-de9e-4e1e-bc07-401aeb2ab42d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:03.238172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:03.239015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:03.240223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 35/71] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:03.241020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc973b1-2e3f-418a-ad1f-0caef5954a17 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.241020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc973b1-2e3f-418a-ad1f-0caef5954a17 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.241230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc973b1-2e3f-418a-ad1f-0caef5954a17 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.242457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 36/72] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:03.244922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc973b1-2e3f-418a-ad1f-0caef5954a17 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.284401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.343040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363bc1b1-fd53-4c64-b29c-992bdece682c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.346952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc973b1-2e3f-418a-ad1f-0caef5954a17 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.357205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.379735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.380465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.380751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.380895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.381255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.381528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.381813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.382092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.391691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:03.395660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e4461305-e133-43ae-be5a-b033654fa01f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:03.396490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:03.397351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:03.398129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:03.399324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.399393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 37/73] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:03.399457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 37/74] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:03.399531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80419-c52b-48d5-8e17-687342c02b47 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.399597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.399779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80419-c52b-48d5-8e17-687342c02b47 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.404974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec80419-c52b-48d5-8e17-687342c02b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.419292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9ec80419-c52b-48d5-8e17-687342c02b47 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:03.420541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9ec80419-c52b-48d5-8e17-687342c02b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:03.504226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 38/75] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:03.504226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.504226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.504226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.504600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.504600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61443528-9e62-4524-9139-5eecd5f5baaf None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.513802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:03.515009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:03.547118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.547737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.548053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.548334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.549006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:03.549370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.550888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.561405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 38/76] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:03.562208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.591839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 39/77] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:03.591839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.634681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.677896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe5ca2-5466-459d-8c93-389d3ac83f7f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.715760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fee2a73-bf4d-48cc-8160-620fa5ccbdb9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.722521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.723833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.724266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.724719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.726001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:03.726491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.728722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.742367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 39/78] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:03.744312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.763317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.763918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.764285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.764564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.771862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.837785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.839942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.859355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da785c6-5570-4296-ac71-8b570b3414c1 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:03.872016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:03.896081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:03.896719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:03.897005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:03.897276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:03.898662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:03.900150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:03.906340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 40/79] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:03.906340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.942850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 40/80] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:03.944150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:03.991695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.004233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480e749c-e306-4514-9fd9-495e772c8795 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.037701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.038270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.038565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.038849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.040012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.041500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.043741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 41/81] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.044528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.052240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b2191-8757-40b2-8884-a4da46b44392 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.082673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.089317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.090093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.090377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.090649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.100685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:04.106010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:04.107062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 41/82] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:04.108063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.147758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b16d8a-d4ed-421a-ac08-7bd1dcb4f3fb None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.183215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.183873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.184157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.184436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.193151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.193151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.193151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 42/83] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.193151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.212830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8322f-214f-418f-b41c-b4a8f386689b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.216583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.248162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.248775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.249058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.249334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.290378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a665335-aba3-4e14-9c71-0a6ccba78f12 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.307682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.317720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.319217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.324045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.324680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.324965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.325242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.326362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.327819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.329472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 43/84] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.330231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.423939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 42/85] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:04.423939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.424580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.431478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542b860d-a6c3-4990-9323-ba43b4cc372d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.466414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.467033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.467317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.467593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.521689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.531069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2cb20-2744-43f4-aa34-acd3682690ef None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.536762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.538236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.564939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.565479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.565763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.566047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.566720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:04.567030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.568442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.577053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 43/86] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.577826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.634226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.644856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 44/87] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:04.645682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.680469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5dc71-5ede-48e5-8ee9-ee2faabc3921 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.714968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.715574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.715886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.716166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.716811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.718280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.743163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4641039-4bc3-4724-a649-4d2c1c44ea99 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.746334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 44/88] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:04 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:04.752450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.766498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.788567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.789465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.789760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.790042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.790959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:04.791716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.793552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.807839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 45/89] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4156f70d4c1744029972dc81f38353a8 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.807839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.891918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a851f5-e272-447a-90be-1f57115e9dae None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:04.917532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:04.944983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:04.946093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:04.946375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:04.946652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:04.949408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:04.951385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:04.953898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 45/90] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:04.954706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:04.964713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20161fd9-531f-408a-a0a4-937c493b094d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.011033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.013020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.013020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.013020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.014214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.016189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.018937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 46/91] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:05.019153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:05.069659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:05.104593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c04b7-22e4-4703-9454-828875655c1f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.154227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f66fb3-ba0b-4b53-a0ce-7b80b06699e0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.158411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.159263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.159713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.160112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.161266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.162730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.164497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 46/92] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:04 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:05.165267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:05.181603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:05.195028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.195648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.195971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.196250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.196985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=0f68be8cf7b24b869bf665ebc09aa5e5) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:05.197347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.199030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.205709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 47/93] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:05 2022] GET /identity/v3/role_assignments?user.id=0f68be8cf7b24b869bf665ebc09aa5e5&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:05.206695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:05.312405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed6078a-afc5-4ee8-a106-500b6f9742aa None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.354557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.355711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.356228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.356718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.398543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93687a9e-3df7-4ebc-b1e4-385033f17478 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.398762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:05.439548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.442507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.519509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.520178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.520434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.520709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.521448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:05.521756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.523431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.530047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 48/94] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:05 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=00b090a41bf543a3a320c3787e8a836d => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:05.578289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 47/95] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:05.583501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:05.650578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:05.703510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1902c84-991f-4c80-9707-8adcdce98477 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.741110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.742146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.742931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.743407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] PATH_INFO: `/v3/projects/8fc8155a57d047ac823e7336ba93fdd8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.744906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] RBAC: Authorizing `identity:get_project(project_id=8fc8155a57d047ac823e7336ba93fdd8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.746793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb175c46-fb3b-409c-a235-25e7e11b566b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.748894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 49/96] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:05 2022] GET /identity/v3/projects/8fc8155a57d047ac823e7336ba93fdd8 => generated 312 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:05.759043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:05.818683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:05.893112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a99850-32c8-4998-8064-15d984ca3823 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:05.928665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:05.929281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:05.929562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:05.929837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:05.930574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=8fc8155a57d047ac823e7336ba93fdd8, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:05.930887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:05.933019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b0cef-723f-4579-9669-cbc4f918b031 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:05.942416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 48/97] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:05 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=8fc8155a57d047ac823e7336ba93fdd8&role.id=4156f70d4c1744029972dc81f38353a8 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:07.546864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942d4d4-9951-41c5-8cbb-ed187c09ec9b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:07.547895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942d4d4-9951-41c5-8cbb-ed187c09ec9b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:07.548406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d942d4d4-9951-41c5-8cbb-ed187c09ec9b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:07.549981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 50/98] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:07.555640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48624-9e32-43ec-a4d1-9eb24587f000 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:07.557073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48624-9e32-43ec-a4d1-9eb24587f000 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:07.557763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da48624-9e32-43ec-a4d1-9eb24587f000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:07.577677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1da48624-9e32-43ec-a4d1-9eb24587f000 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:07.579235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1da48624-9e32-43ec-a4d1-9eb24587f000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:07.684454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 49/99] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:07.879846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db756d-877b-47ce-8930-570b34da9790 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:07.880486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db756d-877b-47ce-8930-570b34da9790 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:07.880992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52db756d-877b-47ce-8930-570b34da9790 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:07.882552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 51/100] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:07.888663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb704c-ff39-44ae-832a-3ed776ae4825 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:07.889179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb704c-ff39-44ae-832a-3ed776ae4825 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:07.889651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb704c-ff39-44ae-832a-3ed776ae4825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:07.912462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-33bb704c-ff39-44ae-832a-3ed776ae4825 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:07.913974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-33bb704c-ff39-44ae-832a-3ed776ae4825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:07.962345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.008538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 50/101] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:08.010023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe659f-a4fc-4aba-9dcb-9a761e6a9b95 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.010517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe659f-a4fc-4aba-9dcb-9a761e6a9b95 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.011012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe659f-a4fc-4aba-9dcb-9a761e6a9b95 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.012572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 52/102] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:08.033375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7595-67b7-41f5-bf46-aed7ff9594c0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.033673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7595-67b7-41f5-bf46-aed7ff9594c0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.033955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e7595-67b7-41f5-bf46-aed7ff9594c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.056693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8c3e7595-67b7-41f5-bf46-aed7ff9594c0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:08.057914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8c3e7595-67b7-41f5-bf46-aed7ff9594c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:08.062750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe659f-a4fc-4aba-9dcb-9a761e6a9b95 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.172797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 51/103] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:08.177113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cedc4-cadf-409e-9171-b9b36858952f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.177417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cedc4-cadf-409e-9171-b9b36858952f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.177683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6cedc4-cadf-409e-9171-b9b36858952f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.179259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 52/104] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:08.189360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.189677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.189927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.234921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:08.236169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:08.285064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe659f-a4fc-4aba-9dcb-9a761e6a9b95 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.324752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.336273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.336904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.337188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.337575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.350068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:08.355396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:08.356266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:08.357039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:08.357605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 53/105] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:08.358940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.375221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 53/106] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:08.376043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.409141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.491084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15befcfc-ab8a-4e42-8ac3-6d73009841eb None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.530838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.531461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.531745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.532069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.533069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:08.533384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:08.534838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d2622-7545-43f6-af80-c702351b35aa None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:08.544304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 54/107] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:08.545549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665085-ad20-4eec-a521-7a7b41306859 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.545937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665085-ad20-4eec-a521-7a7b41306859 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.546180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16665085-ad20-4eec-a521-7a7b41306859 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.547815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 55/108] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:08.552785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665085-ad20-4eec-a521-7a7b41306859 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.590593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c0f09-8a02-42e0-b107-eba31b918e4a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.635495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.636170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.636460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.636740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.638179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:08.640551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6993c-c039-4784-8adf-7ca7496a5454 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:08.642588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 54/109] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:08.647995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.652197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.652564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.671920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.677788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:08.679207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:08.688365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16665085-ad20-4eec-a521-7a7b41306859 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.725881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.726440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.726723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.726998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.728825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:08.730353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:08.733257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.737125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 56/110] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:08.737863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.760038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 55/111] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:08.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.858148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2057d28-1c17-4c8a-b04b-ccef1492ace0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.896517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.897268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.897559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.897902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.902767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c62580-d5b2-4ba7-9e88-75fdc7e65f62 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:08.909629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:08.914964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:08.916068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:08.916838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 57/112] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:08.917470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:08.948791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:08.949777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:08.950290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:08.950656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:08.950776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:08.951480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:08.953135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:08.981252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 56/113] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:46:08 2022] POST /identity/v3/projects => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:08.982169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.034012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.062852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff814a8-ceef-4670-a9fa-3f3235cdc8ee None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.109303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.110149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.111347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.111659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.127094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:09.133207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:09.134199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:09.135547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:09.142460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 58/114] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:09.142460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.146333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f915e-271d-4dc4-94e1-a87d43710e7c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.189685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.190333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.190959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.191250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.192770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.194406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f90e7-8cda-49dd-b904-fce2f074eba1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.196076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 57/115] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:08 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:09.197951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8557a-944a-47d5-ada9-e89af10be270 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.198642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8557a-944a-47d5-ada9-e89af10be270 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.198642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf8557a-944a-47d5-ada9-e89af10be270 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.200102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 58/116] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:09.200522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8557a-944a-47d5-ada9-e89af10be270 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.241240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.262890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0cc21-0775-4d27-8e64-b74293a2a12c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.301081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.301761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.302047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.302328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.303000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:09.303309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.304774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.313046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 59/117] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:09.313925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.318479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.320438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf8557a-944a-47d5-ada9-e89af10be270 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.360089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.360738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.361049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.361295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.373458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:09.378208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:09.379066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:09.379870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 59/118] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:09.385483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.430761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916748d7-76c6-481f-9d08-c0c7b83397a9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.472743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.473384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.473683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.473970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.474630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:09.474940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.476381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9772c8e5-6501-4091-bab0-247bf65e4ce4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.485779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 60/119] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:09.490037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.490585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.490868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.500401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.508971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba782c4-32ea-4497-b74c-3b4b1b8481da None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.512039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:09.513267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:09.547869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.548534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.548824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.549098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/389b0d3b207d47b1af90de57d916552e/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.551057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, user_id=389b0d3b207d47b1af90de57d916552e, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.552873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.566263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 60/120] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:09 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/389b0d3b207d47b1af90de57d916552e/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:09.566263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.602170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.606052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 61/121] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:09.607264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.789407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd0959-d0ee-4803-843b-0cc50de7e897 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.794043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196fbb5-7d40-45a4-9841-83d682ca018e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.838119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.839740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.840245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.840712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.840965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:09.841654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:09.842013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:09.842361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:09.842774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.843192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:09.843572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:09.844807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.845129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:09.846796 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.848320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 61/122] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:09.849907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.857889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 62/123] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:09.858821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:09.950366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:09.967447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a43ad-05d6-4e0c-a353-06a5d5524b21 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:09.976564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2463da-9a40-4fd6-ad79-67980c06213c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.004734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.005378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.005665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.005945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.007093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.008617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.010784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 62/124] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:10.011613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.022386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.023505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.023819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.024099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.026226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.028028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.030488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 63/125] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:09 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:10.031558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.116226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.156166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173eb5a2-f176-4d48-8a91-ae00fda0e6dd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.164355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5580062f-f592-46ab-a4c2-d4341ec7ee1b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.208885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.209983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.210564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.211038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] PATH_INFO: `/v3/users/42b978bb78fa46c0b305ac985a372a29` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.211382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.212087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.212452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.212706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.229721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:10.230135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] RBAC: Authorizing `identity:get_user(user_id=42b978bb78fa46c0b305ac985a372a29)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.231840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.234122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 63/126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/users/42b978bb78fa46c0b305ac985a372a29 => generated 289 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:10.235010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:10.236118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 64/127] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:10.236940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.238291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.258776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.370999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad19037-59af-48a1-8e89-1503fc65c425 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.375670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe10d4-5bb6-4deb-b034-717f6aa2b7b1 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.400969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.415810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.416560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.416853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.417135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/389b0d3b207d47b1af90de57d916552e/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.420188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, user_id=389b0d3b207d47b1af90de57d916552e, role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.422375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.426168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.427755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.428389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.429076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.430826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.432809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.435159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 64/128] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:10.436616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.443092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 65/129] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:10 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/389b0d3b207d47b1af90de57d916552e/roles/00b090a41bf543a3a320c3787e8a836d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:10.444029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.566967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.590892 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c165de-f42a-4529-9664-6f9114d61bdc None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.600467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41488dd-2cf3-4006-8523-535fe8dc506d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.640789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.641508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.641917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.642215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.643995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.645446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.646962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.647443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.648425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:10.648929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.651118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.664016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 65/130] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:10.665745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.718860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.742538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.745070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.787417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435de53-5dbd-427b-a6b9-3170185b6164 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:10.827265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:10.828057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:10.828348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:10.828638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/0f68be8cf7b24b869bf665ebc09aa5e5/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:10.830604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, user_id=0f68be8cf7b24b869bf665ebc09aa5e5, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:10.832097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:10.847372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 66/131] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:10 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/0f68be8cf7b24b869bf665ebc09aa5e5/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:10.848152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.851170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:10.881724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 66/132] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:10.889142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:10.975075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6550c32-6fa4-4ac3-a6b7-9cb63a62d2c3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:11.018312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:11.018963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.019250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.019530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.020718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:11.023850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:11.024870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 67/133] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:11.031084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:11.052195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:11.075204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5b1f7-9c0a-4516-96eb-f3403fc08acf None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:11.120166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:11.156763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac977b-e9d5-4ff5-9c85-20538f56e699 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:11.159151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:11.159834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.160127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.160407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.164502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:11.166670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:11.168607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 67/134] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:10 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:11.174650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:11.203922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:11.204584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.205023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.205318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] PATH_INFO: `/v3/projects/8fc8155a57d047ac823e7336ba93fdd8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.206776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] RBAC: Authorizing `identity:get_project(project_id=8fc8155a57d047ac823e7336ba93fdd8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:11.208364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:11.210683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 68/135] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:11 2022] GET /identity/v3/projects/8fc8155a57d047ac823e7336ba93fdd8 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:11.216996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:11.316083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e268f57-48a5-4f66-91c7-a59e101c5fba None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:11.348383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3993c3-423f-4708-a913-79174551c817 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:11.371828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:11.373985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.375889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.376372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.377320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=42b978bb78fa46c0b305ac985a372a29) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:11.379111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:11.384401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcc94cf-4609-4e06-ae42-da1f7f832514 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:11.395619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 68/136] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:11 2022] GET /identity/v3/role_assignments?user.id=42b978bb78fa46c0b305ac985a372a29&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:11.398603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba7db-02f0-4579-be9d-1a64536f4672 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.402021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba7db-02f0-4579-be9d-1a64536f4672 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.402607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ba7db-02f0-4579-be9d-1a64536f4672 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.405791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 69/137] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:11.411925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4222-f849-4081-98f2-146c23a4cc47 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.412452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4222-f849-4081-98f2-146c23a4cc47 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.412925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4222-f849-4081-98f2-146c23a4cc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.425413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:11.427440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:11.427962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:11.428437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] PATH_INFO: `/v3/projects/8fc8155a57d047ac823e7336ba93fdd8/users/389b0d3b207d47b1af90de57d916552e/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:11.436525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fc8155a57d047ac823e7336ba93fdd8, user_id=389b0d3b207d47b1af90de57d916552e, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:11.438372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:11.457894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:11.464421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ef7b4222-f849-4081-98f2-146c23a4cc47 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:11.465776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ef7b4222-f849-4081-98f2-146c23a4cc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:11.473401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 69/138] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:11 2022] PUT /identity/v3/projects/8fc8155a57d047ac823e7336ba93fdd8/users/389b0d3b207d47b1af90de57d916552e/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:46:11.516284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:11.554371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 70/139] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:12.109339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:12.122115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda16d58-c069-4f51-bd63-93a1e04d3c11 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:12.124734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda16d58-c069-4f51-bd63-93a1e04d3c11 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:12.125321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda16d58-c069-4f51-bd63-93a1e04d3c11 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:12.128056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 71/140] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:12.134275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:12.134820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:12.135296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:12.172138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:12.176802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:12.271979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dad1f0-2604-4585-bbc1-0011a07bcc02 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:12.308788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:12.310733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:12.311250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:12.311722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:12.313216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:12.314972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:12.317210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 70/141] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:12.322691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:12.349098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 72/142] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:12.394236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:12.434419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2079692d-0377-4d8f-b517-5cb73bafb7e9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:12.470909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:12.471828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:12.472318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:12.472793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:12.473637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:12.475287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:12.501048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 71/143] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:12 2022] POST /identity/v3/users => generated 283 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:46:12.895385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:12.996022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:13.027133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827710f-c7b7-4f04-8d3b-0a60711176ce None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:13.082886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:13.083531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.083835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.084120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.100963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-3022e96b-1acb-4810-8b15-3496e8f78327 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:13.105959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:13.106848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:13.108073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 73/144] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:13.114816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:13.259952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f31dac-e050-4181-9893-efdcb923b891 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:13.286552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:13.297343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c384-78ad-449b-be55-9ec937debe9b None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.298030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c384-78ad-449b-be55-9ec937debe9b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.298030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3c384-78ad-449b-be55-9ec937debe9b None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.299288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 74/145] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:13.301146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:13.301984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.302456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.302946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.304625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:13.304772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.305067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.305349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.305692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:13.307120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:13.317604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 72/146] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:13.323652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:13.349822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:13.351066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:13.484049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea5a8b-8584-47dd-809e-3eef11403a04 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:13.502765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 75/147] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:13.526151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:13.526722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.527003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.527283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.528461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:13.529936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:13.531555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 73/148] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:13 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:13.537556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:13.634965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:13.674867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdab774-9bdc-4803-86fb-ea5bfdd534d9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:13.713301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:13.713983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:13.714267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:13.714540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:13.790452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:13.791898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:13.861184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:13.897277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 76/149] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:13.903769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:13.968502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:13.981313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:14.058161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b52a7-db19-4b3a-aca3-2e14d3829174 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:14.099912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:14.100551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.100828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.101422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.103080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:14.104566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83716eb3-ce3b-4a44-85bc-f4cc5b63b509 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:14.106224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 74/150] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:13 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:14.107647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77a25-5cd4-4f7d-a653-32b5f4a64c72 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.107961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77a25-5cd4-4f7d-a653-32b5f4a64c72 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.108237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77a25-5cd4-4f7d-a653-32b5f4a64c72 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.109439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 75/151] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:14.116002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.116588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.116588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.137488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:14.138760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:14.182463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537063a-81e0-44ac-9e50-83acd23aa419 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:14.226032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:14.227261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 76/152] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:14.228107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.228246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:14.228724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.229233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.250551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:14.257234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:14.258686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:14.259487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 77/153] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:14.274531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:14.294059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:14.311184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:14.355998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e15c7-375c-4d2d-997c-5050264b2fde None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:14.401943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:14.402891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.403170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.403490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.404294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:14.404604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:14.406337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:14.409716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bace4e00-3355-42b7-b65e-388e8c7a8b52 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:14.413561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 77/154] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:14 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4156f70d4c1744029972dc81f38353a8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:14.466635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:14.467581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.468086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.468576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.469465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:14.469982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:14.471656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabc48-8b99-4845-b464-8e4e1dca74aa None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:14.481895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 78/155] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:14.487693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:14.537771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec9b65-4bd4-4151-8be9-90796b152ec7 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.538419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec9b65-4bd4-4151-8be9-90796b152ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.538976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec9b65-4bd4-4151-8be9-90796b152ec7 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.541056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 79/156] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:14.546975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b985adc-8416-4151-aa9d-0435aad404cf None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.547543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b985adc-8416-4151-aa9d-0435aad404cf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.548040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b985adc-8416-4151-aa9d-0435aad404cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.582834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3b985adc-8416-4151-aa9d-0435aad404cf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:14.587940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3b985adc-8416-4151-aa9d-0435aad404cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:14.731091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a80154-0f4f-4f30-a06c-e141c1077e59 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:14.764462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:14.776043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 80/157] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:14.778532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dc2d2-54a8-4c3d-b625-e23c10156b9b None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.779080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dc2d2-54a8-4c3d-b625-e23c10156b9b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.780774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040dc2d2-54a8-4c3d-b625-e23c10156b9b None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.783978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 81/158] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:14.794676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.794959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.799809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.813853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:14.814506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:14.814788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:14.823576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:14.824916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:14.826403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:14.830375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 78/159] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:14 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:14.845340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:14.846210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:14.849997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:14.883286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:15.004564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 82/160] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:15.005656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.074032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5609d5-1162-45ad-8e3c-ed98382dcb73 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.158375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:15.159039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:15.159266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:15.159531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:15.175133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5bcaf4-1917-4b88-855f-ffb95caa5282 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.236818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:15.238385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:15.241094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:15.242463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:15.257158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:15.264691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:15.265628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:15.266481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:15.269467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 83/161] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:15.278756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.279982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:15.319434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:15.320997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:15.340178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:15.471616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 79/162] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:15.472627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.492020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c54456-1ef7-45e5-87cf-7adaa605db05 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.557722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:15.558753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:15.559291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:15.559763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:15.560724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:15.561241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:15.562896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:15.580470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 84/163] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:15.581577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.611161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:15.661695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51031b-337a-4440-b2ee-41e95ed415ad None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.678727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:15.705188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f023dde7-6ef9-4c33-bf46-e7faa6eeadac None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.732552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:15.733125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:15.733401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:15.733669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:15.742367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:15.743113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:15.743514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:15.743928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:15.745235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:15.746834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:15.751196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 85/164] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:15.751196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.752313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:15.757597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:15.758472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:15.759416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:15.761361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 80/165] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:15.767822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:15.932200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4e134e-cd39-4f3c-80fa-6316c5abc541 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:15.968435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd58ea-daf0-4fee-8a90-f165a13b4432 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.008167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.009277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.009698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.010112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] PATH_INFO: `/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.016514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] RBAC: Authorizing `identity:get_user(user_id=0f68be8cf7b24b869bf665ebc09aa5e5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.018172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.019961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 86/166] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/users/0f68be8cf7b24b869bf665ebc09aa5e5 => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.020943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.041533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.042198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.042478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.042756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.045058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.071559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:16.076385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:16.077276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:16.078063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 81/167] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:16.084613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.095138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.190591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e778e76-ad9b-4e48-ba4b-8e9f940c8a52 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.230127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.231164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.231643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.232188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.233225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:16.233759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.235536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.242996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 87/168] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=00b090a41bf543a3a320c3787e8a836d => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.244095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.322483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c2d5cd-3041-4f94-a7f5-ec64eeef9c7c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.379524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.413710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.414344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.414621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.414891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.415573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:16.415896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.417315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.434812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 82/169] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.437119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13faf8-9d78-493d-8ae6-b90714280a09 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.439482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.498118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.498734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.499010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.499279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.548203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.601287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4944b0f-c27d-44cd-8274-43d0d1835095 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.622141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.623994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf4041-20d6-4feb-a8f7-7669c07d41a8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.648620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.649247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.649526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.651764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.651764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:16.651764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.652195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.664181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 83/170] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.664965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.718594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.758132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 88/171] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:16.759745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b5bb-b1f0-42d3-9c75-6c30cf8e8e31 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.760267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b5bb-b1f0-42d3-9c75-6c30cf8e8e31 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.760733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07b5bb-b1f0-42d3-9c75-6c30cf8e8e31 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.762219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 89/172] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:16.763364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b5bb-b1f0-42d3-9c75-6c30cf8e8e31 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.789142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea357a1e-3149-4fc3-9273-334219198776 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.827927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.829047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.829549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.830084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] PATH_INFO: `/v3/users/389b0d3b207d47b1af90de57d916552e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.831582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] RBAC: Authorizing `identity:get_user(user_id=389b0d3b207d47b1af90de57d916552e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.833362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.835351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 84/173] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/users/389b0d3b207d47b1af90de57d916552e => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.836699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:16.864381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:16.889922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07b5bb-b1f0-42d3-9c75-6c30cf8e8e31 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.933638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.934276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.934698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.935094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] PATH_INFO: `/v3/users/42b978bb78fa46c0b305ac985a372a29` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.936277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] RBAC: Authorizing `identity:get_user(user_id=42b978bb78fa46c0b305ac985a372a29)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.938537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1239b-b00e-4f2e-865f-b94c963f750b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.940325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 90/174] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/users/42b978bb78fa46c0b305ac985a372a29 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.942059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776208de-3489-47a0-a022-625428e2f34b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.942341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776208de-3489-47a0-a022-625428e2f34b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.942610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776208de-3489-47a0-a022-625428e2f34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.948393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3efdc3-19b6-4c28-b271-ffded622d03a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:16.963249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-776208de-3489-47a0-a022-625428e2f34b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:16.964801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-776208de-3489-47a0-a022-625428e2f34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:16.989160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:16.990064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:16.990349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:16.990631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:16.992503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:16.994558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:16.996582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 85/175] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:16.997519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.000427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.046075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 91/176] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:17.046932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776208de-3489-47a0-a022-625428e2f34b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.066618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.109733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e8829-0e1a-4568-9725-12ba2c976dd7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.143679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.144349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.144626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.144895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.160070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776208de-3489-47a0-a022-625428e2f34b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.197700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.198342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.198624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.198899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.200316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.201726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.203782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 92/177] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:17.209237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.213936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.215415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d9c7c-4218-438a-a9f6-c51b4fd9ac42 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.246852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.276680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.316055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837ce5-3162-4eb4-9302-73e1ac1acf26 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.319089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 86/178] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:17.320976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9345-412f-48ea-8a51-fd8d8e5b0f9c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.321261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9345-412f-48ea-8a51-fd8d8e5b0f9c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.321527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e9345-412f-48ea-8a51-fd8d8e5b0f9c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.322910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 87/179] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:17.323776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4638bb1-0b79-449d-a20c-9b9e2640fe52 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.324075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4638bb1-0b79-449d-a20c-9b9e2640fe52 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.324342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4638bb1-0b79-449d-a20c-9b9e2640fe52 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.325641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 88/180] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:17.326200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4638bb1-0b79-449d-a20c-9b9e2640fe52 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.354020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.354625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.354902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.355173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.355930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=0f68be8cf7b24b869bf665ebc09aa5e5) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:17.356236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.357699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.363278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 93/181] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/role_assignments?user.id=0f68be8cf7b24b869bf665ebc09aa5e5&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:17.364086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.438727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4638bb1-0b79-449d-a20c-9b9e2640fe52 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.479156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.479862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.480144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.480416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/42b978bb78fa46c0b305ac985a372a29/roles/4ebc708580ce4aaab8f3df1c71cdfeb2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.482361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, user_id=42b978bb78fa46c0b305ac985a372a29, role_id=4ebc708580ce4aaab8f3df1c71cdfeb2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.483870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.495031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.500074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea4004c-e16f-4ff2-bf83-af43d04a89bb None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.500227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 89/182] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:17 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/users/42b978bb78fa46c0b305ac985a372a29/roles/4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:17.501141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.533699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.575775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.576753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.577040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.577741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.592533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-540c9446-0008-4015-add4-ac790e840da1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:17.601207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:17.602194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:17.602988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 94/183] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:17.605716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.605716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.605716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.628627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:17.629879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:17.650498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fab4e-4a59-4022-a165-0d073b06086e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.696680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.697233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.697512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.697785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] PATH_INFO: `/v3/projects/8fc8155a57d047ac823e7336ba93fdd8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.698933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] RBAC: Authorizing `identity:get_project(project_id=8fc8155a57d047ac823e7336ba93fdd8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.700406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c974d7-7ccd-49fd-8bb6-5fcca57d8c8d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.703847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 90/184] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/projects/8fc8155a57d047ac823e7336ba93fdd8 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:17.703847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.703847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.703847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.721368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 95/185] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:17.722215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.727382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:17.728849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:17.820058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 91/186] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:17.821100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:17.824947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.844974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104753ed-73c2-476b-a9c7-51e673893247 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:17.854860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:17.921452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:17.922064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:17.922339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:17.922608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:17.925216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:17.925700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:17.928235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:17.946333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 96/187] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:17.951971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.011546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a64a4-8ead-4ea3-a1df-a2ae947f2d4e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.059774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.060421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.060705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.061401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.062427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=8fc8155a57d047ac823e7336ba93fdd8, user.id=389b0d3b207d47b1af90de57d916552e) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:18.062740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.064943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.073107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 92/188] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/role_assignments?user.id=389b0d3b207d47b1af90de57d916552e&scope.project.id=8fc8155a57d047ac823e7336ba93fdd8&role.id=4156f70d4c1744029972dc81f38353a8 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:18.074954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.101410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd9a9-6081-40a5-b854-9dccfcfafb0b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.152188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.152881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.153159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.153428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.175270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.177090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.180212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 97/189] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:17 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:18.181311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.218544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.220204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.245505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85ec64-a30f-45ad-82a1-28a9280e893d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.287922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.288137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.288391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.288558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.289675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.291124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.294686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 93/190] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:18.295285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.318695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd16e-1f9d-462f-9384-0490a685806d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.360728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.361371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.361653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.361928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.373650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:18.378578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:18.379469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 98/191] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:18.381066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.418020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d1bd7-79c8-4db0-ad57-d03158def056 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.451208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.451877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.452118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.452389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.468995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.475846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.505900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e94fc4-8929-4404-aabc-2db291aed607 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.529754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.531212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.544535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.545145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.545419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.545718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.546334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.547889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.580177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 99/192] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:18 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:18.581084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.675125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 94/193] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:18.683849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.716768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9a2f2-9278-4dd0-b83e-4a806226bdd7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.744601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.749233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:18.759524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.760779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.760779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.760779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.761400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:18.761705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.763134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.773127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 100/194] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:18.788471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.817455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd06669-65e5-4881-a967-6b9029f98b56 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.860728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.861544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.861835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.862117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.863258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.865022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.871669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 95/195] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:18.872641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:18.914825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658a3fd-1ad7-4d6f-a4ed-57c38d5d1206 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:18.967600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:18.968413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:18.968690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:18.968958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:18.970102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:18.971558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:18.973200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 101/196] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:18.979512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:19.014914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f5f65-e3c4-40c5-900a-64612097ee24 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:19.043210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:19.052039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:19.060847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:19.062224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.062700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.063210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.064421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=510c82edb9544c67a7bd9d06cf0ed32a) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:19.064960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:19.067431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:19.075968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 96/197] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/role_assignments?user.id=510c82edb9544c67a7bd9d06cf0ed32a&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:19.109609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd202039-f3dd-4300-a1f7-6c1361631365 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:19.155528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:19.156184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.156466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.156738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.258115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:19.259733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dea253d-4955-40f7-8977-caf18c3bb3e9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:19.335947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:19.394793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 102/198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:19.401538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:19.520716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f948a-5793-42a2-9e7b-d1bb62e7ec95 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:19.565433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:19.566126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.566413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.566690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.572699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:19.574054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578dee6-6fc6-42d5-a69b-a0e28da9096d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:19.574162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f996b4-017d-4b17-b703-ab2364d131fa None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.574562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f996b4-017d-4b17-b703-ab2364d131fa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.574741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f996b4-017d-4b17-b703-ab2364d131fa None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.576224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 97/199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:19 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:19.576348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 103/200] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:19.581254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.581544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.581727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f996b4-017d-4b17-b703-ab2364d131fa None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:19.581821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.606057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:19.607174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:19.734602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f996b4-017d-4b17-b703-ab2364d131fa None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:19.736436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:19.779445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 98/201] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:19.793524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:19.794139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:19.794422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:19.794695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:19.795428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=510c82edb9544c67a7bd9d06cf0ed32a) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:19.795733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:19.797216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:19.803621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 104/202] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:19 2022] GET /identity/v3/role_assignments?user.id=510c82edb9544c67a7bd9d06cf0ed32a&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=00b090a41bf543a3a320c3787e8a836d => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:20.103095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:20.215836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a2638-c727-47ef-9de6-06c2fde6ea37 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:20.258250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:20.258893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:20.259174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:20.259472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:20.273434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:20.278205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:20.279082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:20.279917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 99/203] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:20.286936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:20.324215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:20.411952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee1df2-3264-4b85-aabb-37f24f7105b0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:20.460167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:20.463835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:20.472896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 105/204] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:20.479621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:20.526753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:20.586440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc514530-1845-40e3-940e-5063fca38aad None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:20.622180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:20.622819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:20.623100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:20.623371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] PATH_INFO: `/v3/users/42b978bb78fa46c0b305ac985a372a29` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:20.624560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] RBAC: Authorizing `identity:get_user(user_id=42b978bb78fa46c0b305ac985a372a29)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:20.626031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:20.627825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 100/205] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:20 2022] GET /identity/v3/users/42b978bb78fa46c0b305ac985a372a29 => generated 289 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:20.633619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:20.729049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:20.741814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e127e4e-ed9f-4905-80c8-0058478c9c54 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:20.778642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:20.779258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:20.779541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:20.779838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:20.855587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:20.857128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:20.951310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:20.979412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 106/206] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:20.985279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:21.114288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5da682-62a1-4577-b788-769b868c396f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:21.133365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:21.166931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:21.167641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.167961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.168237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.169418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:21.170934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49c9a3-9353-408c-86ff-b6f0d89a148e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:21.172978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 101/207] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:20 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:21.179369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:21.305476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ceacdf-da47-4136-9585-903a5441b98f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:21.323895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93df2-86db-4ae5-9016-0ee45b8cf2ef None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.324190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93df2-86db-4ae5-9016-0ee45b8cf2ef None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.324452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d93df2-86db-4ae5-9016-0ee45b8cf2ef None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.325826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 102/208] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:21.330686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d541-0852-4bdd-bdbd-8b2ef828f752 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.330971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d541-0852-4bdd-bdbd-8b2ef828f752 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.331249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d541-0852-4bdd-bdbd-8b2ef828f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.362855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4727d541-0852-4bdd-bdbd-8b2ef828f752 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:21.365604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4727d541-0852-4bdd-bdbd-8b2ef828f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:21.375621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:21.376328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.376608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.376880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.377674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa, user.id=42b978bb78fa46c0b305ac985a372a29) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:21.377989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:21.379438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0af380-137c-4030-a65d-0b0dec484e23 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:21.386937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 107/209] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:21 2022] GET /identity/v3/role_assignments?user.id=42b978bb78fa46c0b305ac985a372a29&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:21.402359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8098-7521-4c2a-8734-6c7f16c21f3c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.402687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8098-7521-4c2a-8734-6c7f16c21f3c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.402904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf8098-7521-4c2a-8734-6c7f16c21f3c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.404649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 108/210] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:21.410121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec321c4-c478-4fc7-b1ad-3a6995a90f1a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.410396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec321c4-c478-4fc7-b1ad-3a6995a90f1a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.410669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec321c4-c478-4fc7-b1ad-3a6995a90f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.430086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-dec321c4-c478-4fc7-b1ad-3a6995a90f1a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:21.431319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-dec321c4-c478-4fc7-b1ad-3a6995a90f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:21.496249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:21.548436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 109/211] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:21.559078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 103/212] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:21.830021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86ba8c-8f10-4e54-9633-8cc08a1cf2f0 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.830500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86ba8c-8f10-4e54-9633-8cc08a1cf2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.830500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86ba8c-8f10-4e54-9633-8cc08a1cf2f0 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.831893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 110/213] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:21.837721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:21.838013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:21.838295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:21.843618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86ba8c-8f10-4e54-9633-8cc08a1cf2f0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:21.866509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:21.867705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:21.939054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 104/214] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:21.939935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:21.970335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86ba8c-8f10-4e54-9633-8cc08a1cf2f0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:21.986075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.024100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.024738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.025071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.025343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.027025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.028956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.031035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 111/215] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:22.036415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.055122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428775c-8e25-4ce0-bdb3-d651780ae439 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.087242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.092212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.092850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.093126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.093394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.104964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-94638503-50a3-4a9e-82e4-931b03afe821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:22.111185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:22.112274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:22.113092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 105/216] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:22.114368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ace9-76cb-43e7-8dbc-80643337e62b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.114368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ace9-76cb-43e7-8dbc-80643337e62b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.114586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074ace9-76cb-43e7-8dbc-80643337e62b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.119852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 106/217] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:22.119852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074ace9-76cb-43e7-8dbc-80643337e62b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.153460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf863ebc-5871-4fd5-a9fa-5eb568830d8c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.188852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.189454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.189730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.189998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.190645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.192112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae3c0b3-aed6-4533-8311-f02147d19da4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.213949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 112/218] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:22 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:46:22.215600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.215919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.216221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.235314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.235520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:22.236899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:22.242522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074ace9-76cb-43e7-8dbc-80643337e62b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.294488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.295427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.295427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.295698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.296373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:22.296673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.298096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.308897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 107/219] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:22.309841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.324550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 113/220] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:22.325350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.368649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.462873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdc279-e5fe-41fc-92be-35bccb5ccb49 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.473556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c6ae7-46ee-4323-acbe-adea3172b142 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.503880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.505650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.505650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.505900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.521182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:22.523735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.524417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.524695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.524968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] PATH_INFO: `/v3/users/77c17cc09b474992b7165debd7f3586d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:22.527001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:22.527868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:22.528689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 108/221] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:22.537926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.541007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] RBAC: Authorizing `identity:get_user(user_id=77c17cc09b474992b7165debd7f3586d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.542564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.544314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 114/222] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/users/77c17cc09b474992b7165debd7f3586d => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:22.551395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.564763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.672375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.684639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ced34-60cc-42eb-bc30-904e946a8cc6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.689290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af52c05a-41e0-4ea4-8ce6-543ad9a8cdfe None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.725628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.726597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.727024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.727445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.728319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:22.728768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.730305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.731153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.731771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.732102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.732738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.744237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 109/223] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:22.745342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.818290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:22.820047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:22.865204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.889833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e630b66-90fa-493c-afe9-df281f5fcfb3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:22.926881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:22.929780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:22.930594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:22.932053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:22.932053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:22.943998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 115/224] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:22.944794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:22.944975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:22.950905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:22.951837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:22.952660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:22.955333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 110/225] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:22.956458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.073091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d90fd-f2ea-4cfb-b265-8da831d800b9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.076917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4886bc-ae2c-49d8-bcb9-0d66c5d64a87 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.111851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.112489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.112766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.113037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.114201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.114427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.115940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.116480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.116859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.117176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.122821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.122957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 116/226] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.122957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.124667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.126883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 111/227] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:22 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.128657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.151075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:23.196448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:23.242448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4327f5-3240-4919-a728-02b0d190805b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.247104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d91fd-7e32-4b04-9592-24c8be8992ad None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.278240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.279100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.279479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.280146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.281199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.282733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.284518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 112/228] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:23 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.285603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.286920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.287357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.287782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.288653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:23.289077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.290724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.291030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.300880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 117/229] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.301915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.376008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:23.399602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fafa36-2a34-49be-9c81-a131ff90ba93 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.415051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8f2f-16f0-4b02-b549-4c1b7ff0e95c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.434599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:23.437774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.438716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.438834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.439219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.440904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=77c17cc09b474992b7165debd7f3586d) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:23.441316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.443771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.454808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 113/230] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:23 2022] GET /identity/v3/role_assignments?user.id=77c17cc09b474992b7165debd7f3586d&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4156f70d4c1744029972dc81f38353a8 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.454808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.483115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.484176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.484629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.485089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/510c82edb9544c67a7bd9d06cf0ed32a/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.490477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, user_id=510c82edb9544c67a7bd9d06cf0ed32a, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.492329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.530288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 118/231] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:23 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/510c82edb9544c67a7bd9d06cf0ed32a/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:23.578718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f198852-f603-46eb-9b68-f0cd30bd2f62 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.636310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.636967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.637249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.637528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.644748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.647233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.649634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 114/232] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:23 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.655240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.788473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a56b0d-2c9d-41b0-8813-75d0b1c45d64 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.826227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:23.827287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:23.827768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:23.828264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:23.829607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:23.831246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61010808-0e74-4614-b9e5-05a963ae402c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:23.834060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 119/233] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:23 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:23.839647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:23.859017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:23.963480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fefcb0-fcbb-49b9-9b67-ea4c315e60ad None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:23.999748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:24.000448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:24.000730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:24.001007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/510c82edb9544c67a7bd9d06cf0ed32a/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:24.003245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, user_id=510c82edb9544c67a7bd9d06cf0ed32a, role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:24.004747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ab6dc-ffbe-4297-a501-d534129c6eec None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:24.035052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 115/234] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:23 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/510c82edb9544c67a7bd9d06cf0ed32a/roles/00b090a41bf543a3a320c3787e8a836d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:24.495228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0d4d-2315-4b4e-bc2e-f653ea90d8d3 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:24.496191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0d4d-2315-4b4e-bc2e-f653ea90d8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:24.496669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cd0d4d-2315-4b4e-bc2e-f653ea90d8d3 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:24.498520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 120/235] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:24.503383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:24.503671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:24.503976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:24.533012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:24.534207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:24.600242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f224a8d-8c6d-41d8-8bf8-db92c72d4dbf None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:24.600560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f224a8d-8c6d-41d8-8bf8-db92c72d4dbf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:24.600790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f224a8d-8c6d-41d8-8bf8-db92c72d4dbf None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:24.602386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 121/236] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:24.611989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:24.612380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:24.612529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:24.642645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:24.643271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:24.643957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:24.648368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 116/237] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:24.703245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:24.730721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 122/238] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:25.122299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:25.138589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:25.285697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41f9c94-fb0e-4f1e-b423-87f3bc0e59cc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:25.303839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2910d5d-2470-4cfa-b37e-1656ec8fb10e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:25.332102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:25.333474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:25.333834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:25.334116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:25.335242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:25.336833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:25.338732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 123/239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:25.344738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:25.373073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:25.373631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:25.373912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:25.374194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:25.393944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:25.400904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:25.401889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:25.403771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 117/240] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:25.407744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:25.408715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:25.528570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:25.556493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd6779-8e8d-4061-9cd4-cde14521bf11 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:25.591163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f5509b-8939-4af0-ac9d-cd36944ce0bf None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:25.644985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:25.648982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:25.649267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:25.649546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:25.650242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:25.650548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:25.651999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:25.660154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:25.661626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:25.664348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:25.664812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:25.668096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:25.670698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:25.676287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 118/241] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:25.682751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:25.701081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 124/242] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:25 2022] POST /identity/v3/users => generated 293 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:25.805416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe62c-20ae-4fa6-94c0-dc2d57c9eb20 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:25.834833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:25.855524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:25.856642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:25.857124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:25.857601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] PATH_INFO: `/v3/users/92de10b76e9144cfa3b17c48659ec2b6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:25.878834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] RBAC: Authorizing `identity:get_user(user_id=92de10b76e9144cfa3b17c48659ec2b6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:25.881448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e32c9-d67a-4bcd-954c-f7a9da60d0ad None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:25.883776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 119/243] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:25 2022] GET /identity/v3/users/92de10b76e9144cfa3b17c48659ec2b6 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:25.889712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.011952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991578bf-cf97-4764-ab46-1ebe6293e62e None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.012551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991578bf-cf97-4764-ab46-1ebe6293e62e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.013050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991578bf-cf97-4764-ab46-1ebe6293e62e None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.014632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 120/244] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:26.022269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2f8c-606e-409d-9eea-f5cbafab5dab None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.022735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2f8c-606e-409d-9eea-f5cbafab5dab None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.023227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc2f8c-606e-409d-9eea-f5cbafab5dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.043230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:26.046484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f137a1-3e86-4c43-b670-c83ef4ed1541 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.048459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-00bc2f8c-606e-409d-9eea-f5cbafab5dab None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:26.050078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-00bc2f8c-606e-409d-9eea-f5cbafab5dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:26.094635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:26.095384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.095741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.096126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.150991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 121/245] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:26.169455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8ff8-bf8c-410c-a5ca-1e357faf2df0 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.169760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8ff8-bf8c-410c-a5ca-1e357faf2df0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.170016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd8ff8-bf8c-410c-a5ca-1e357faf2df0 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.175202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 122/246] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:26.177274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.177670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.177948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.180043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:26.181528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:26.196425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:26.197630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:26.282100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 123/247] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:26.305660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:26.309731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 125/248] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:26.315165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.366276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:26.432889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f938112-fbf2-4519-a188-4aa627d05e8d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.470837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:26.471478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.471758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.472067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.473275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:26.475390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:26.477388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 124/249] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:26 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:26.483071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.542562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.608395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea65ace9-5cbc-4a66-8860-810428444305 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.655390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:26.656021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.656297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.656566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.657297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=92de10b76e9144cfa3b17c48659ec2b6) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:26.657594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:26.659033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:26.667088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 126/250] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:26 2022] GET /identity/v3/role_assignments?user.id=92de10b76e9144cfa3b17c48659ec2b6&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:26.667957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.691842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c4cf0b-ac41-48f4-8b24-71d21bcdcf58 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.720181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:26.758851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:26.759514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.759813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.760096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.783904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:26.792325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:26.793381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:26.794194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:26.796804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 125/251] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:26.806060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:26.824165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:26.896712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e298382-88c0-42eb-a3a3-9d2552f50d64 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.965769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:26.967125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:26.967957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:26.968414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:26.977012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ce9be-ae8a-4fe9-b99e-474e84c83cf5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:26.986572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-157fbe30-d0d9-4062-8e28-21edec9d84ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:26.993454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:26.994391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:26.995206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:26.998771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 127/252] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:27.000427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e1a4-e598-4e05-b963-25fd099b9636 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.000920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e1a4-e598-4e05-b963-25fd099b9636 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.001397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e1a4-e598-4e05-b963-25fd099b9636 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.003281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 128/253] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:27.004939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e1a4-e598-4e05-b963-25fd099b9636 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.055539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.056115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.056400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.056675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.057356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:27.057657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.059044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08611e46-fb5f-45bf-abb8-c0d4429b7218 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.078994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 126/254] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:27.080263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90521-e49b-455d-978d-d842a516180f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.080552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90521-e49b-455d-978d-d842a516180f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.080823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd90521-e49b-455d-978d-d842a516180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.120810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5fd90521-e49b-455d-978d-d842a516180f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:27.122038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5fd90521-e49b-455d-978d-d842a516180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:27.128073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b6e1a4-e598-4e05-b963-25fd099b9636 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:27.170889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:27.177540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:27.209610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.209610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.209870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.210144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.211024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:27.211024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.216853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.221667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 129/255] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:27.222278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.253174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 127/256] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:27.254040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd90521-e49b-455d-978d-d842a516180f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.408975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd90521-e49b-455d-978d-d842a516180f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:27.425740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b76f25d-3e9e-41b6-8bb6-4e19f3353e90 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:27.447986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.448979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.449484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.449955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.451315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.453007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.454942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 128/257] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:27.461734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.493721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:27.528501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.529151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.529435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.529712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] PATH_INFO: `/v3/users/77c17cc09b474992b7165debd7f3586d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.530886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] RBAC: Authorizing `identity:get_user(user_id=77c17cc09b474992b7165debd7f3586d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.532395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.534304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 130/258] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/users/77c17cc09b474992b7165debd7f3586d => generated 297 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:27.539706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.583501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793a72f-a01f-47ab-9011-000ad63bedf2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:27.647256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.648389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.648673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.648943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.687481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:27.797768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.799470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.822040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e9d31-4c14-4e47-857f-0a2310e86029 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:27.852117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:27.904508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:27.905168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:27.905450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:27.905726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:27.906897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:27.908395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:27.911897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 131/259] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:27.912773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:27.936679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 129/260] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:27.937880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.050423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0677c-49a6-4baf-a18c-2bdd252ae821 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.081478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:28.092118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.093089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.093572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.094061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/77c17cc09b474992b7165debd7f3586d/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.096270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, user_id=77c17cc09b474992b7165debd7f3586d, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:28.098008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:28.119642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 130/261] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:27 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/77c17cc09b474992b7165debd7f3586d/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:28.120803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.151155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf7daa7-c292-4f9d-988c-4db3daa248f7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.200017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:28.224703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.225364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.225646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.225919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.252100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:28.258399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:28.259279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:28.260132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 132/262] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:28.269547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.375994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f090016-4bfd-4727-99b5-602f47547970 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.468391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f51a1-cdec-4792-9d58-587ad9e8611c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.469432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.470079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.470355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.470626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.476537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:28.478007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:28.479730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 131/263] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:28 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:28.489589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.509557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.510194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.510473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.510745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.511826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:28.512144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:28.513609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:28.526407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 133/264] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:28.531282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.586712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:28.601632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:28.682936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f2b346-34d1-4a84-89ac-d77b3799b377 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.733029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86dc8ca-1ed9-4f05-a7d7-dd3d42a466d2 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:28.761575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.762499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.762883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.763309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.764590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=510c82edb9544c67a7bd9d06cf0ed32a) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:28.765124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:28.767406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03acb7b-c427-48dd-a952-f05b5f485f6a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:28.781028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 132/265] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:28 2022] GET /identity/v3/role_assignments?user.id=510c82edb9544c67a7bd9d06cf0ed32a&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:28.848324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:28.849038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.849554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.850124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] PATH_INFO: `/v3/users/510c82edb9544c67a7bd9d06cf0ed32a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.851371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] RBAC: Authorizing `identity:get_user(user_id=510c82edb9544c67a7bd9d06cf0ed32a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:28.851477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86797-30c8-44a1-8f57-47a9460b0c70 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.851899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86797-30c8-44a1-8f57-47a9460b0c70 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.851969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86797-30c8-44a1-8f57-47a9460b0c70 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.852931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:28.853323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 133/266] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:28.856043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 134/267] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:28 2022] GET /identity/v3/users/510c82edb9544c67a7bd9d06cf0ed32a => generated 283 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:28.858357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab6d3-049a-409e-a106-1b46c6a66c0a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:28.858642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab6d3-049a-409e-a106-1b46c6a66c0a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:28.858917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ab6d3-049a-409e-a106-1b46c6a66c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:28.860727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:28.904987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-714ab6d3-049a-409e-a106-1b46c6a66c0a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:28.906210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-714ab6d3-049a-409e-a106-1b46c6a66c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:29.025564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 134/268] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:29.046598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323436f-1fc2-4085-88d5-d3c06d7e7d0d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.065530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ab15-b77e-4ca8-a91e-5a85878d18be None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.066139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ab15-b77e-4ca8-a91e-5a85878d18be None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.067200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a7ab15-b77e-4ca8-a91e-5a85878d18be None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.069079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 135/269] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:29.072170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.076539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.077097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.077529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.099641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:29.101545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:29.120344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.121323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.122597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.123084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.187321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.208578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.210083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.221707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 136/270] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:29.342531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 135/271] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:29.352247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.382033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.398834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.500986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801e4de-8a33-4901-a27c-cf865cf112c7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.539630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f9fb48-fee5-4330-afd2-abe53403ada7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.541340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.541985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.542261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.542526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.564905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:29.570081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:29.570960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:29.571760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:29.575880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 136/272] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:29.596235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.597355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.608203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.608464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.610816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.612411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.614137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.620301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 137/273] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:29.625147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.686392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.692119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.721937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801d2a9-9a4d-44ab-be87-edbcc7849860 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.748283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1f26f4-fe6c-4056-af6b-e859af0debaf None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.762590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.763285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.763561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.763867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.764534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:29.764834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.766225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.775209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 137/274] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:29.776038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.793959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.794740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.795023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.795293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.796401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.797840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ff4f1-0e33-4fcf-add3-92b01fd37ad0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.799555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 138/275] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:29.801107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a2c6-51e3-4244-8d99-de8bb0959c46 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.801387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a2c6-51e3-4244-8d99-de8bb0959c46 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.801653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384a2c6-51e3-4244-8d99-de8bb0959c46 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.802976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 139/276] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:29.803539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a2c6-51e3-4244-8d99-de8bb0959c46 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.894728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade11965-818b-4133-89fc-9c0a96486d7b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.916717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384a2c6-51e3-4244-8d99-de8bb0959c46 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:29.931997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.932602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.932883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.933166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.933898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=510c82edb9544c67a7bd9d06cf0ed32a) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:29.934188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.934754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.935634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.937714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:29.948036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 138/277] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/role_assignments?user.id=510c82edb9544c67a7bd9d06cf0ed32a&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=00b090a41bf543a3a320c3787e8a836d => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:29.949522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:29.956124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:29.956337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.956575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.956918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] PATH_INFO: `/v3/users/568523fc191c4250b096c9c29aab957d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.973478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] RBAC: Authorizing `identity:get_user(user_id=568523fc191c4250b096c9c29aab957d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:29.974967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af03152-40a5-4524-b79c-aad844978c45 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:29.976993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 140/278] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:29 2022] GET /identity/v3/users/568523fc191c4250b096c9c29aab957d => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:29.978023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:29.978296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:29.978570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:29.998141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:29.999426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:30.087814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 141/279] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:30.087814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:30.109415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf2ac6-de0b-46ff-aede-dd38c5984674 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:30.148705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:30.149261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.149537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.149809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.150520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:30.152018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:30.185771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 139/280] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:29 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:30.219615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fa5c38-70b5-4430-a637-2d82cad980a6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:30.223163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:30.266252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:30.266880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.267155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.267424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.268212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, user.id=568523fc191c4250b096c9c29aab957d, scope.system=all) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:30.268514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:30.270970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:30.277814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 142/281] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 24 09:46:30 2022] GET /identity/v3/role_assignments?user.id=568523fc191c4250b096c9c29aab957d&scope.system=all&role.id=4156f70d4c1744029972dc81f38353a8 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:30.411341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:30.473592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:30.522966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0f49e4-2495-4fe4-bc07-7596a16986b1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:30.561502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:30.562411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.562841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.563287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.575491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-16516aae-26ee-4475-a3a9-d0b5c1ed7a0d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:30.581036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:30.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:30.583513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:30.584578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 140/282] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:30.594750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:30.711763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cea7874-f0a3-4dd1-a030-2375accf1dfd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:30.713923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032e774-5f0f-4760-a7af-bf1fbb57e8e4 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.714213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032e774-5f0f-4760-a7af-bf1fbb57e8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.714492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032e774-5f0f-4760-a7af-bf1fbb57e8e4 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.719637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 141/283] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:30.728538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.729127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.729760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.749892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:30.751491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:30.764819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:30.765436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.765720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.765992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.766661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:30.766964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:30.768373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:30.778212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 143/284] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:30.786568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:30.835038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 142/285] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:30.839858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:30.929749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a4701d-a9bf-44b3-9e94-960ff065b011 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:30.970901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:30.971458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:30.971737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:30.972067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] PATH_INFO: `/v3/users/92de10b76e9144cfa3b17c48659ec2b6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:30.973288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] RBAC: Authorizing `identity:get_user(user_id=92de10b76e9144cfa3b17c48659ec2b6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:30.974717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:30.976623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 144/286] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:30 2022] GET /identity/v3/users/92de10b76e9144cfa3b17c48659ec2b6 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:30.982108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.041018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:31.101212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad154e1e-6fd8-4762-a867-a079eef2fcf3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.137566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.138204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.138484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.138814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:31.140033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:31.141486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:31.143204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 143/287] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:30 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:31.149167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.159581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.263235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47189ea9-2cad-46a2-baac-ef954e2846bd None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.284849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:31.299218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.299882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.300471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.300760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:31.322013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:31.328450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:31.329330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:31.330141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 144/288] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:31.338582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.395634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89b9c1-c00b-4b0b-b99a-b6d23e5f199c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.397947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:31.472556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.473533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.474015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.474487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/92de10b76e9144cfa3b17c48659ec2b6/roles/4ebc708580ce4aaab8f3df1c71cdfeb2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:31.484618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, user_id=92de10b76e9144cfa3b17c48659ec2b6, role_id=4ebc708580ce4aaab8f3df1c71cdfeb2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:31.486705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:31.509507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19362-543d-4faf-adb2-a595aaef494d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.542947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 145/289] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:46:31 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/users/92de10b76e9144cfa3b17c48659ec2b6/roles/4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:31.556566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.557222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.557502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.557778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:31.558528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:31.558836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:31.560486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:31.576219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 145/290] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:31.582509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.742504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021ed45-f174-44d6-a769-93fa460365f9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.780987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.781625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.781904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.782178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] PATH_INFO: `/v3/users/77c17cc09b474992b7165debd7f3586d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:31.783362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] RBAC: Authorizing `identity:get_user(user_id=77c17cc09b474992b7165debd7f3586d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:31.785518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:31.787463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 146/291] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:31 2022] GET /identity/v3/users/77c17cc09b474992b7165debd7f3586d => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:31.793760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:31.833888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:31.917844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616d2e7-1b42-4150-9f68-0163194a078c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:31.965845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:31.966692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:31.967135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:31.967542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.059232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:32.060964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:32.102512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:32.193475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 146/292] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:32.199228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:32.237645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:32.350863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abba1cd-5aed-4248-a80d-8c5de1479c80 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:32.400640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:32.401657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.402165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.402641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.408633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:32.410366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3072ac1e-24eb-435e-bf01-12f63733c4a9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:32.412306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 147/293] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:32 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:32.418103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:32.580937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fffe77-ca44-4425-8a63-8d75d9b8a4e5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:32.626937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:32.627475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.627759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.628728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.629467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=77c17cc09b474992b7165debd7f3586d) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:32.629776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:32.631223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584049b3-7731-497b-8d56-6442f21515f1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:32.640354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:32.640494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 147/294] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:32 2022] GET /identity/v3/role_assignments?user.id=77c17cc09b474992b7165debd7f3586d&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4156f70d4c1744029972dc81f38353a8 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:32.748036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a0c3-0881-4e8c-b292-1b25aa9c9b9a None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.748723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a0c3-0881-4e8c-b292-1b25aa9c9b9a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.749208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a0c3-0881-4e8c-b292-1b25aa9c9b9a None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.750838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 148/295] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:32.757927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e37cf-5eae-44c2-8dd7-5e4f2376cbeb None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.758229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e37cf-5eae-44c2-8dd7-5e4f2376cbeb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.758480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5e37cf-5eae-44c2-8dd7-5e4f2376cbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.777969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cf5e37cf-5eae-44c2-8dd7-5e4f2376cbeb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:32.779197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cf5e37cf-5eae-44c2-8dd7-5e4f2376cbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:32.860163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 148/296] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:32.873823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412412b-f9fa-4ac4-9e40-ab026091b9f2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.874115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412412b-f9fa-4ac4-9e40-ab026091b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.874398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412412b-f9fa-4ac4-9e40-ab026091b9f2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.876051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 149/297] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:32.881539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.881765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.882038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.900595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:32.901780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:32.940683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d25d-3dd0-4ce9-8028-299f838e98f2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.941244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d25d-3dd0-4ce9-8028-299f838e98f2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.941443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30d25d-3dd0-4ce9-8028-299f838e98f2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.943086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 150/298] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:32.953544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:32.953831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:32.954111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:32.979078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:32.981733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:32.991914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 149/299] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:33.069810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 151/300] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:33.156660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.222144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.270230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d1ac3-2d83-4544-a87e-a65de06ddced None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.306465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d4d41-4666-479a-a122-4017e879807f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.307105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4d41-4666-479a-a122-4017e879807f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.307385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4d41-4666-479a-a122-4017e879807f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.307752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4d41-4666-479a-a122-4017e879807f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.309071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.319154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717d4d41-4666-479a-a122-4017e879807f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-717d4d41-4666-479a-a122-4017e879807f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:33.324315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:33.325191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:33.325999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 150/301] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:33.333029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d4d41-4666-479a-a122-4017e879807f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.351523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.435262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e61b-6a6e-4861-b461-c4868cfabc94 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.448463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d4d41-4666-479a-a122-4017e879807f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.469904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.477943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.478642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.478885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.479116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.480244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:33.481673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:33.483448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 152/302] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:33.484082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.493700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.494315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.494599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.494877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.495557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:33.495893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:33.497299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:33.506503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 151/303] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:33.507226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.588882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.599745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db79b3-f304-4195-a0e9-f264bc907d7e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.614359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb359c2-8bde-483c-84a2-4b1ec68f5560 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.635744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.636408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.636688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.636963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.649313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:33.652403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.653014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.653296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.653664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] PATH_INFO: `/v3/groups` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:33.654085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:33.654918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:33.655837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:33.656473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:33.656542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 153/304] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:33.656542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.661626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:33.679832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 152/305] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Aug 24 09:46:33 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:33.680536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.739774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.805494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5d685-fc69-482d-bf2d-dc8b02387dd6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.818010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d57a01-8123-4de6-beab-f88c3f79da2a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.858174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.859183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.859668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.860166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] PATH_INFO: `/v3/users/568523fc191c4250b096c9c29aab957d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.860290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:33.861133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:33.861417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:33.861678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:33.863061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:33.863367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:33.863849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=568523fc191c4250b096c9c29aab957d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:33.864787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:33.866755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:33.869872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 154/306] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/users/568523fc191c4250b096c9c29aab957d => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:33.879822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 153/307] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:33.879989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.882199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:33.907957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.995761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e716c-62e6-4382-bf21-b7c24a2a1383 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:33.997217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:33.999840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502ed95-b592-40f2-8130-4a96d605e8d3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:34.033160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.034790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.035276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.035749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] PATH_INFO: `/v3/users/72fd70e1988b455ebd3dc18a3559c4d2` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.037176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.038091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.038370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.038643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] PATH_INFO: `/v3/system/users/568523fc191c4250b096c9c29aab957d/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.040077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=568523fc191c4250b096c9c29aab957d, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:34.041452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8671f-54b7-4d53-a90e-3659b8f644f2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:34.051134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 155/308] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Aug 24 09:46:33 2022] PUT /identity/v3/system/users/568523fc191c4250b096c9c29aab957d/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:34.067814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29b8a-065b-463a-8bdf-43deb651c0b4 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.067814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29b8a-065b-463a-8bdf-43deb651c0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.067814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a29b8a-065b-463a-8bdf-43deb651c0b4 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.067814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 156/309] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:34.068200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd70e1988b455ebd3dc18a3559c4d2)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:34.068273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.068348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:34.068425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.068425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.068555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 154/310] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:33 2022] GET /identity/v3/users/72fd70e1988b455ebd3dc18a3559c4d2 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:34.071357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:34.082843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:34.084444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:34.173049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 157/311] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:34.210675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7090e-60c7-40af-83bd-b7fd65b0fed4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:34.254815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.255423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.255712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.256035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.256770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, user.id=72fd70e1988b455ebd3dc18a3559c4d2, scope.system=all) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:34.257079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:34.258542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:34.264366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 155/312] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 24 09:46:34 2022] GET /identity/v3/role_assignments?user.id=72fd70e1988b455ebd3dc18a3559c4d2&scope.system=all&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:34.448765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:34.465746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:34.560975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae7bc-349a-4368-b63b-67f923924d57 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:34.596862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.597497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.597777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.598048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.608601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:34.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:34.615019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:34.615820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 158/313] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:34.621052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:34.648646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:34.730274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134b09b-9ae4-4086-ae76-02cfc931b8a1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:34.766230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.766842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.767183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.767461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.768808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:34.769244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:34.770631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:34.779628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 156/314] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:34.785536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:34.870137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:34.896923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea87ed-0ddf-4a27-8701-eca99b1e6dd7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:34.934582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:34.935220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:34.935500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:34.935774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] PATH_INFO: `/v3/users/92de10b76e9144cfa3b17c48659ec2b6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:34.936946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] RBAC: Authorizing `identity:get_user(user_id=92de10b76e9144cfa3b17c48659ec2b6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:34.938409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:34.940955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 159/315] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:34 2022] GET /identity/v3/users/92de10b76e9144cfa3b17c48659ec2b6 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:34.946574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:35.037765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:35.050389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951edc-3021-4052-a88b-3a8827f7ec13 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:35.086223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:35.086836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:35.087117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:35.087390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:35.157667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:35.160033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:35.256499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:35.265735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 157/316] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:35.276938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:35.386252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7dd3a-0179-40be-a1d7-8a4053a4e1d6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:35.423861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:35.425318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:35.425773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:35.426221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:35.427715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:35.429422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920de98-3978-442c-aa55-a0af71e7bb59 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:35.435904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 160/317] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:35 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:35.437665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:35.545023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad7288-39f8-4158-b720-047fcc0e178a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:35.582594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:35.583208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:35.583489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:35.583761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:35.584514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6, user.id=92de10b76e9144cfa3b17c48659ec2b6) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:35.584819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:35.586265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa36e3f-7350-40de-950c-79efee9cdcdd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:35.597308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 158/318] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:46:35 2022] GET /identity/v3/role_assignments?user.id=92de10b76e9144cfa3b17c48659ec2b6&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:35.793965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034aca-be98-46c1-9fb9-d02c3b4bd931 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:35.794460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034aca-be98-46c1-9fb9-d02c3b4bd931 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:35.794460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08034aca-be98-46c1-9fb9-d02c3b4bd931 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:35.796281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 161/319] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:35.801901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf454a-6f43-4f70-8fa7-cf16139f7c0c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:35.802474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf454a-6f43-4f70-8fa7-cf16139f7c0c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:35.802934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf454a-6f43-4f70-8fa7-cf16139f7c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:35.823033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9ecf454a-6f43-4f70-8fa7-cf16139f7c0c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:35.824700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9ecf454a-6f43-4f70-8fa7-cf16139f7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:35.876155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:35.898939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 159/320] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:36.127358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355de4-b4e6-4781-ac99-8d6005e51734 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.127682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355de4-b4e6-4781-ac99-8d6005e51734 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.127901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13355de4-b4e6-4781-ac99-8d6005e51734 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.129395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 162/321] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:36.134423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77e3-a6f2-4b09-b955-c6f7d365cf6a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.134860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77e3-a6f2-4b09-b955-c6f7d365cf6a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.135016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b77e3-a6f2-4b09-b955-c6f7d365cf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.154050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d15b77e3-a6f2-4b09-b955-c6f7d365cf6a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:36.155256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d15b77e3-a6f2-4b09-b955-c6f7d365cf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:36.166804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13355de4-b4e6-4781-ac99-8d6005e51734 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.178991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:36.226008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 160/322] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:36.229111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88527e31-e230-41ba-873c-de1a3e10b6b1 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.229584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88527e31-e230-41ba-873c-de1a3e10b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.229874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88527e31-e230-41ba-873c-de1a3e10b6b1 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.231268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 161/323] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:36.236962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.237256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.237527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.255297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:36.256528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:36.273950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13355de4-b4e6-4781-ac99-8d6005e51734 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.308723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.309417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.309657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.309884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.310974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:36.312461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:36.316229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 163/324] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:36.320191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.331252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 162/325] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:36.400197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:36.422175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a285ece-554a-47a8-ba1b-0de94b60e8bd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.462995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.463604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.463909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.464184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] PATH_INFO: `/v3/groups` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.464803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:36.466235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:36.483954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 164/326] 10.222.0.78 () {60 vars in 1134 bytes} [Wed Aug 24 09:46:36 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:46:36.508128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.556159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:36.607544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.629973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886b19b2-0906-459d-b6c8-6ea45f4cd99c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.669249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.669812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.670092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.670370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.688916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:36.694410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:36.695287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:36.696365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 163/327] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:36.702470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.743128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bc1f-3341-4f79-94ee-e78c550a6d09 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.785728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.786562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.787163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.787441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.801719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:36.807165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:36.808473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:36.809788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 165/328] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:36.811226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:36.814707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.833393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:36.837619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df507-6da8-461a-89aa-2688ec2817e2 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.902136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.902136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.902136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.902136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.902523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:36.902523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:36.904273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:36.915125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 164/329] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:36.920845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:36.954323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb128122-5e78-4141-b4d9-86b5b24aba51 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:36.991731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:36.992386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:36.992666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:36.992939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:36.993609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:36.993913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:36.995312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:37.004927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 166/330] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:37.010423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:37.058617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a680a97a-c78e-4672-a7d9-240e8e10d5f6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:37.095383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:37.107305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:37.108085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:37.108404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:37.108756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] PATH_INFO: `/v3/users/568523fc191c4250b096c9c29aab957d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:37.114306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] RBAC: Authorizing `identity:get_user(user_id=568523fc191c4250b096c9c29aab957d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:37.116076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:37.118260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 165/331] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:36 2022] GET /identity/v3/users/568523fc191c4250b096c9c29aab957d => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:37.125545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:37.164553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff6095-8a86-4a97-9c3b-982c0c4ea0d3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:37.168995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:37.204590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:37.205771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:37.206095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:37.206364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] PATH_INFO: `/v3/users/72fd70e1988b455ebd3dc18a3559c4d2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:37.207500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd70e1988b455ebd3dc18a3559c4d2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:37.208986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:37.210569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 167/332] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:37 2022] GET /identity/v3/users/72fd70e1988b455ebd3dc18a3559c4d2 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:37.216364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:37.278156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182b3ebf-8a69-466a-b615-05a345826b37 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:37.334283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:37.334902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:37.335178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:37.335449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:37.336275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] RBAC: Adding query filter params (role.id=4156f70d4c1744029972dc81f38353a8, user.id=568523fc191c4250b096c9c29aab957d, scope.system=all) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:37.336579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:37.338042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8afb735-27d4-4857-b845-10d4c15b842a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:37.358455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df29c4f-27b7-4ec5-a572-0670793b9852 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:37.360508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 166/333] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 24 09:46:37 2022] GET /identity/v3/role_assignments?user.id=568523fc191c4250b096c9c29aab957d&scope.system=all&role.id=4156f70d4c1744029972dc81f38353a8 => generated 510 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:37.393178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:37.403917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:37.404835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:37.405262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:37.405682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] PATH_INFO: `/v3/system/users/72fd70e1988b455ebd3dc18a3559c4d2/roles/4ebc708580ce4aaab8f3df1c71cdfeb2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:37.407495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=72fd70e1988b455ebd3dc18a3559c4d2, role_id=4ebc708580ce4aaab8f3df1c71cdfeb2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:37.410103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d01e13-0440-4cbe-8cde-e12ef7d694b8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:37.419236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 168/334] 10.222.0.78 () {58 vars in 1334 bytes} [Wed Aug 24 09:46:37 2022] PUT /identity/v3/system/users/72fd70e1988b455ebd3dc18a3559c4d2/roles/4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:38.803653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b55c-a173-4f18-b033-2ff3b9c7814e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:38.804165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b55c-a173-4f18-b033-2ff3b9c7814e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:38.804165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b55c-a173-4f18-b033-2ff3b9c7814e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:38.805526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 167/335] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:38.811067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd9ee-707f-4073-8103-9ffc4953ba2c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:38.811946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd9ee-707f-4073-8103-9ffc4953ba2c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:38.812443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6fd9ee-707f-4073-8103-9ffc4953ba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:38.836822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-7c6fd9ee-707f-4073-8103-9ffc4953ba2c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:38.838489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7c6fd9ee-707f-4073-8103-9ffc4953ba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:38.929238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 169/336] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:39.094492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36efd8-2430-448f-acd9-74da70dc7f69 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:39.095108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36efd8-2430-448f-acd9-74da70dc7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:39.095840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da36efd8-2430-448f-acd9-74da70dc7f69 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:39.097406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 168/337] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:39.107027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:39.107581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:39.108100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:39.136828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:39.138358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:39.229461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 170/338] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:39.481899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36efd8-2430-448f-acd9-74da70dc7f69 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:39.603674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:39.621650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da36efd8-2430-448f-acd9-74da70dc7f69 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:39.667306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:39.667977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:39.668256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:39.668526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:39.686443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:39.691774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:39.692688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:39.693515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 169/339] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:39.699101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:39.769137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:39.828705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9445ae-345c-43f9-9a1b-f26d2ced4d8e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:39.856972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:39.902360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:39.902978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:39.903248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:39.903513 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:39.904212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:39.904508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:39.905894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:39.924601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 171/340] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:39.931413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.035169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6379a-237c-40f4-88b0-73c13df0409a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.067839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b5cb7-9d6e-4939-83fd-4420229c4fc4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.075221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.075899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.076177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.076444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.088795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.099605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-5328a9c6-7597-4dc9-b854-a12ac03ff0d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:40.105171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:40.107255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:40.109268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 170/341] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:40.111190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8ead3-5f0d-4df5-97d8-5481db5da9d0 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8ead3-5f0d-4df5-97d8-5481db5da9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d8ead3-5f0d-4df5-97d8-5481db5da9d0 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 171/342] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d836d-ef76-4351-b043-7b0a52e07d9d None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d836d-ef76-4351-b043-7b0a52e07d9d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.115838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d836d-ef76-4351-b043-7b0a52e07d9d None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.116708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.117361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.117361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.117645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 172/343] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:40.117761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.118962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b600ac6-0e40-4979-85ee-d271c477dcb3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.119345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b600ac6-0e40-4979-85ee-d271c477dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.119863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b600ac6-0e40-4979-85ee-d271c477dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.139825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4b600ac6-0e40-4979-85ee-d271c477dcb3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:40.141135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4b600ac6-0e40-4979-85ee-d271c477dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:40.144123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.195642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.197132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.236829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 173/344] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:40.238100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.238385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.238660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.257829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:40.259008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:40.317779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 172/345] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:40.318647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.337024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.339426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 174/346] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:40.340161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.430964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.441164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbba61-0ab7-4430-a75e-95a6fa1a96c3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.476058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3f6d24-3feb-4594-a0a0-4bb9621d85dd None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.483526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.484430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.485185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.486330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.487375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:40.487923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.489592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.501891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 173/347] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:40.506307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.518060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.519134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.519415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.519683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.520889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.522705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.524994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 175/348] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:40.531240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.601106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.632168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ed466-baa0-4226-9d6e-5097be3f97f5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.665316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f73a5d-1ba4-4050-853c-0b6b4748d074 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.672918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.673859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.674380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.674876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.707383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.708489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.708881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.709174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.713808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.718076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.719587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.727972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 176/349] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:40.728845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.763140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.764691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.898495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:40.937825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 174/350] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:40.938762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:40.941448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f36a3b-e413-4d42-9655-c7b35cc492af None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:40.983635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:40.984288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:40.984836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:40.985109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:40.986168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:40.986481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:40.987957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:40.994113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 177/351] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4156f70d4c1744029972dc81f38353a8 => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:40.994829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.054478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49bd4b1-6aa0-4ec1-9fd1-1cf25ee85140 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.074323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.101131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.101781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.102055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.102326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.119037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:41.122817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673b3891-163d-418b-b1fd-19be0ceb0c52 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:41.125006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:41.125942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:41.127648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 175/352] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:41.129621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.167007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.167575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.167887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.168163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.169315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.170760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37390326-27f3-4c3f-a74b-90ad9cdde390 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.177694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 178/353] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:40 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.179048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea7720-2213-4254-aa2a-6c1fdfaacb66 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.179334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea7720-2213-4254-aa2a-6c1fdfaacb66 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.179610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea7720-2213-4254-aa2a-6c1fdfaacb66 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.181241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 179/354] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:41.182032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea7720-2213-4254-aa2a-6c1fdfaacb66 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.244880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0aa5cd-0820-4a36-b4a7-23c818f78a26 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.280782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.281416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.281693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.281959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.294588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:41.298381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea7720-2213-4254-aa2a-6c1fdfaacb66 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:41.299937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:41.302254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 176/355] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:41.304162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.334369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.340157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.340772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.341059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.341398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.342313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:41.342798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.344478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd7721-783b-42df-a304-f6c6d7a2058e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.354076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 180/356] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.355520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.355828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.356110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.375968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:41.377914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:41.419663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561470d-b2d2-4aa7-ab9f-1bc9fddc8c98 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.455172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 181/357] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:41.456342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.457378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.458016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.458291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.458668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.459952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.461476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.463161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 177/358] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.463919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.464949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.576144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7cbd-8f10-4b25-b5a9-c09aecc18c42 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.584585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68a3f39-c2bf-4a5b-b30f-4ecb68bb7e68 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.601928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.618607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.619234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.619885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.620174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.621124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:41.621433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.623300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.630073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.630917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.631334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.631763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] PATH_INFO: `/v3/users/72fd70e1988b455ebd3dc18a3559c4d2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.633096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] RBAC: Authorizing `identity:get_user(user_id=72fd70e1988b455ebd3dc18a3559c4d2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.635310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 182/359] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.635310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.636625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.638821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 178/360] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/users/72fd70e1988b455ebd3dc18a3559c4d2 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.639777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.700248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.750234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b5b66-d2c2-4cd8-a002-c328f2dab1d8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.789143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.789767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.790051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.790466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.791208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:41.791544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.792980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.799882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 183/361] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=00b090a41bf543a3a320c3787e8a836d => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.801363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.865106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:41.921578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0fd25-eddf-443e-8238-3c87834a11fe None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.945319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9b3af-825d-4f9f-85fe-907078e37b05 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:41.962563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.963114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:41.963375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:41.963649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:41.964427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] RBAC: Adding query filter params (role.id=4ebc708580ce4aaab8f3df1c71cdfeb2, user.id=72fd70e1988b455ebd3dc18a3559c4d2, scope.system=all) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:41.964738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:41.966206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:41.981502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 184/362] 10.222.0.78 () {56 vars in 1308 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/role_assignments?user.id=72fd70e1988b455ebd3dc18a3559c4d2&scope.system=all&role.id=4ebc708580ce4aaab8f3df1c71cdfeb2 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:41.982440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:41.998673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:41.999952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.000457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.001594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.111110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e62f7-8ce3-40e0-b07f-80005f1696eb None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.113248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.114737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabda4c0-bbf0-4c85-9000-632cf531db90 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.124195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.178634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.179290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.179571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.179868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.180165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.205331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-cfa20f9a-ce22-4873-9522-27ffff8c4b21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:42.210855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:42.211760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:42.212688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 185/363] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:42.214480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7ede-1068-4f9c-bd6c-7d5f42191430 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.214772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7ede-1068-4f9c-bd6c-7d5f42191430 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.215048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e7ede-1068-4f9c-bd6c-7d5f42191430 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.216913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 186/364] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:42.222002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e7ede-1068-4f9c-bd6c-7d5f42191430 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.303040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 179/365] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:42.304734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.305277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.305739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.327505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:42.329257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:42.383666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e7ede-1068-4f9c-bd6c-7d5f42191430 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.421270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 180/366] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:42.422440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.425170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.425865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.426323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.426474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.427253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:42.428465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.429118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.439970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 187/367] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:42.445005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.488712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.509046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.551645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd9e3-32a6-41a1-b3ac-959e32671eda None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.568348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738c434-df35-4782-91d5-60a243df38d4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.589464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.590264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.590641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.591304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.592275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.593812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.596822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 181/368] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:42.602260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.607591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.608390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.608806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.609502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.683272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.684981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.713782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740414c9-b05e-4999-907f-75e5a3fc8c3b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.716798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.748450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:42.755782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.756591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.756960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.757336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.758588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.760590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.762210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 182/369] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:42.768394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.798632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 188/370] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:42.806107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.896471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6269c32-b47d-4aad-9d8a-0a70ec28442a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.933299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:42.933914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:42.934196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:42.934474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:42.935207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:42.935518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:42.937017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:42.945452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 183/371] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4156f70d4c1744029972dc81f38353a8 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:42.949449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b08ac-a093-4f76-b7f2-a0d550d6bac4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:42.993978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:42.999399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:43.010168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.010826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.011111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.011463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.017258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:43.020107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:43.022425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 189/372] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:43.031274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:43.111061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:43.181044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdb56c-5b12-4b6b-9e28-751799e3e828 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:43.224523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc2d90-8b33-43a0-adf1-500f6c79f74b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:43.234720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.235396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.235672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.237732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.254164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:43.261002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:43.262036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:43.263265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 184/373] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:43.268519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:43.317805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.318457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.318738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.319007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.320200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:43.321650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:43.323255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 190/374] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:43 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:43.330009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:43.381670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e4c40-34c6-439c-b13a-b686ab17e67a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:43.405389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:43.419590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.420233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.420509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.420833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.421497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:43.421843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:43.423801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:43.432569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 185/375] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:43.437846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:43.480112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:43.545483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd6968c-b9d1-4549-8e80-c0ded7378532 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:43.586760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.587372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.587650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.587949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.588676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:43.588979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:43.590418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b11c4-f40f-4b34-aae9-a70b94bac550 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:43.599019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 191/376] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:43 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=00b090a41bf543a3a320c3787e8a836d => generated 260 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:43.601782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e1b05f-7fa6-4cb1-834d-6ad1e5040b3e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:43.673619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:43.674363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.674703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.674979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.760858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:43.782846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:43.784359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:43.792935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c29b3-6a8c-486c-af8b-db63556eee36 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.793271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c29b3-6a8c-486c-af8b-db63556eee36 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.793555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c29b3-6a8c-486c-af8b-db63556eee36 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.795455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 192/377] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:43.803481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:43.804178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:43.804466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:43.831061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:43.832279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:43.918044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 193/378] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:43.976686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 186/379] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:43.982494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.073001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:44.102352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee115e-039a-48b5-9ad2-9ac1bda9ab4f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:44.139468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:44.140154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.140517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.140787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] PATH_INFO: `/v3/projects/7790bbe18b09475b81076e89944d2568` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.142873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] RBAC: Authorizing `identity:get_project(project_id=7790bbe18b09475b81076e89944d2568)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:44.144384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:44.146345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 194/380] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:43 2022] GET /identity/v3/projects/7790bbe18b09475b81076e89944d2568 => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:44.151577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.286387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:44.303674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9119c2f2-e2ad-447b-87bc-9b8a1bd4ce79 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:44.343938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:44.344499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.344774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.345066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] PATH_INFO: `/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.354896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] RBAC: Authorizing `identity:get_group(group_id=3232bd8430f6458a8168f2fb87e9e2d8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:44.356401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:44.358365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 187/381] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:44.364966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.467457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.525800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff51e105-ce74-4163-8e65-37040476caf8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:44.555435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:44.565376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:44.565907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.566252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.566539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.567331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] RBAC: Adding query filter params (group.id=3232bd8430f6458a8168f2fb87e9e2d8, role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=7790bbe18b09475b81076e89944d2568) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:44.567649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:44.569121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c872228-33a8-4629-a334-1637d7dd216e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:44.575409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 195/382] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/role_assignments?group.id=3232bd8430f6458a8168f2fb87e9e2d8&scope.project.id=7790bbe18b09475b81076e89944d2568&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:44.578278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c15ec-7c48-4fa2-9ccb-0b9d65140cf0 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.578278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c15ec-7c48-4fa2-9ccb-0b9d65140cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.578278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c15ec-7c48-4fa2-9ccb-0b9d65140cf0 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.579184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 196/383] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:44.585686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.586031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.586305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.607821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:44.608958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:44.615929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82f77d-943d-4d60-a4e2-38a0c3bcddcb None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:44.657111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:44.658100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.659124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.659584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.673788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:44.680247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:44.681259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:44.682089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:44.684951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 188/384] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:44.692107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 197/385] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:44.693266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.734585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:44.819094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c4866-99a2-47ef-b506-efa733e2a30a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:44.856949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:44.857887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:44.858336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:44.858774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:44.859606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] RBAC: Adding query filter params (name=member) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:44.860087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:44.861883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:44.871470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 189/386] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:44.876883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:44.999199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.019704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c994812-4944-41ac-a438-27f66c32b53e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.046479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:45.071581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.072274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.072555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.072825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.075271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.077053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.082529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 198/387] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:45.089139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.126403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:45.160083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7178f5c-2e19-4127-a363-78cbe6da2224 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.229709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.230811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.231504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.232635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.253188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:45.260415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:45.261337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:45.262137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 190/388] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:45.267834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.304456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7938687b-2256-4e79-90f0-760cff58d913 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.374519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.375169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.375451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.375724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.377034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.379335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.382325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 199/389] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:45 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:45.386454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.460885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d04c0-0d73-49e1-a4eb-2b228069d65b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.475257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:45.507581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:45.522694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.523106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937b7d50-7360-42c2-b234-a7edd6276bc3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.523747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.524201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.524613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.525451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:45.525896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.527441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.542160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 191/390] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:45.547814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.564004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.564923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.565310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.565727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/groups/420ad112404e4739b2440ef502b8d1e7/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.568582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, group_id=420ad112404e4739b2440ef502b8d1e7, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.570497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730892c-c823-4eb5-8316-ec910ca1f489 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.596403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 200/391] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 24 09:46:45 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/groups/420ad112404e4739b2440ef502b8d1e7/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:46:45.611999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3a91-e888-42f6-8d62-ef0107c2154e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.612431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3a91-e888-42f6-8d62-ef0107c2154e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.612818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5c3a91-e888-42f6-8d62-ef0107c2154e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.614620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 201/392] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:45.620454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.620739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.621014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.641558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:45.642437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:45.689684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722e5dc8-6a48-4b84-b73f-12198867c2f3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.732134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 202/393] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:45.743087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.743783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.744095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.744368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.745546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.747024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.749127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 192/394] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:45 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:45.757301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:45.826291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:45.898413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0eddf-b6e5-4a62-9896-e6642361e943 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:45.935719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:45.936726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:45.937224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:45.937704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:45.939140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:45.940898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:45.943986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 203/395] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:45 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:45.952490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.032790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:46.090460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84ee29-7525-48b1-8775-72b009a335e4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:46.128689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:46.129361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.129640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.129915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/groups/420ad112404e4739b2440ef502b8d1e7/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.132768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=78e064f8e56e4c4a94dedba92fcf78aa, group_id=420ad112404e4739b2440ef502b8d1e7, role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:46.134223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6636f247-44b8-4158-a65f-a3c532bd70b0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:46.151508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 193/396] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 24 09:46:45 2022] PUT /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa/groups/420ad112404e4739b2440ef502b8d1e7/roles/00b090a41bf543a3a320c3787e8a836d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:46:46.185331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.222882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff07b1-2498-40b2-910d-75ef9f2ff512 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.223215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff07b1-2498-40b2-910d-75ef9f2ff512 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.223546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff07b1-2498-40b2-910d-75ef9f2ff512 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.225631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 194/397] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:46.235934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.236263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.236489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.273849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:46.275478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:46.328658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572708cd-d2e1-48ac-bfe8-dca6b82ccd52 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:46.351100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:46.376898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:46.378464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.378912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.379388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.394955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:46.400144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:46.401048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:46.401925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 204/398] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:46.409702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 195/399] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:46.410614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.539313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a80e97-ce45-458a-bf7c-46b956929080 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:46.583144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:46.583828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.584134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.584412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.585184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:46.585516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:46.586907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:46.600619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 205/400] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:46.606742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.668414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:46.734883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db168336-5b11-46b8-aa4a-b77cb926a77d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:46.773958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:46.774536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.774817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.775096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.776265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:46.777722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:46.779710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 196/401] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:46 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:46.785442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.831069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:46.872098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.912133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189adf1-2855-44aa-9525-7b7f1489ce89 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:46.953281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:46.953855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:46.954138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:46.954410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:46.955646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:46.957243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:46.958918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 206/402] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:46 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:46.965383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:46.993813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6a4f3-38a4-42e7-98c6-86ad597b2a6c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.014355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:47.035091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.035937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.036229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.036688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.049585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:47.054872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:47.055933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:47.056750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 197/403] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:47.065016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:47.159766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900594ab-2806-4d62-a2d0-aa60627fd459 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.170801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:47.206227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fceb3-ecfc-40c1-94b8-22843a7388a0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.245916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.246790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.247235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.247663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.248502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:47.248973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:47.250511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:47.253551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.254233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.254516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.254792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/groups/420ad112404e4739b2440ef502b8d1e7/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.256782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, group_id=420ad112404e4739b2440ef502b8d1e7, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:47.258226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c70a4-25ae-4f66-8428-18388aa6d501 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:47.261870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 198/404] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:47.267475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:47.296595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 207/405] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 24 09:46:46 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/groups/420ad112404e4739b2440ef502b8d1e7/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:47.330553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:47.403223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93707e0d-ac5b-4e48-b91b-22d9009bce4d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.441352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.442245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.442675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.443092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.452593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:47.454325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:47.461515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 199/406] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:47 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:47.465784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd9926-0ee4-4bb8-935d-4bc28b82582f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.466235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd9926-0ee4-4bb8-935d-4bc28b82582f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.466676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd9926-0ee4-4bb8-935d-4bc28b82582f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.468233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 208/407] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:47.469372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:47.485811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.486099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.486383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.530523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:47.531966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:47.586043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfb7a8-db13-419d-9525-b9fa57591d4b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.597008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:47.629944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.630955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.631395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 209/408] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:47.631675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.631992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.633217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:47.635482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:47.637523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 200/409] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:47 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:47.642958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:47.749821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:47.756868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7755b8c-517e-40f3-aa77-5c1f9fbd7a0a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:47.808735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:47.809435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:47.809717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:47.809994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/groups/420ad112404e4739b2440ef502b8d1e7/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:47.812418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] RBAC: Authorizing `identity:create_grant(project_id=eb3b7ee937974dc182e3a2e76819d7c6, group_id=420ad112404e4739b2440ef502b8d1e7, role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:47.813934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:47.844665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 210/410] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 24 09:46:47 2022] PUT /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6/groups/420ad112404e4739b2440ef502b8d1e7/roles/00b090a41bf543a3a320c3787e8a836d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:46:48.006583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:48.138863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ca1f2-2c9c-42b1-a55d-0122fafc1711 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:48.177323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:48.177960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.178242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.178520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.191660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-99d9cd26-2079-48bd-bc30-87a1e6893368 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:48.197041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:48.197935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:48.199740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 201/411] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:48.204950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:48.324341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda3e14-e100-4bae-b441-a3ee7afc9647 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:48.361811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0807e-909c-4bae-9214-38ddc734e0a4 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.362124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0807e-909c-4bae-9214-38ddc734e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.362361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0807e-909c-4bae-9214-38ddc734e0a4 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.363728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 202/412] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:48.363854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:48.364414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.364694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.365028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.365713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:48.366015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:48.367401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:48.374702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.374996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.375277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.377540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 211/413] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:48.382986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:48.404228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:48.416831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:48.418672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:48.491754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:48.542282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8554f02c-87f2-4b57-a7c9-75560fd8473f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:48.561929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 203/414] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:48.587912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:48.588482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.588764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.589102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] PATH_INFO: `/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.590894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] RBAC: Authorizing `identity:get_group(group_id=3232bd8430f6458a8168f2fb87e9e2d8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:48.592391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:48.594267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 212/415] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:48 2022] GET /identity/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8 => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:48.599810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:48.727069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562e3bf-4ef9-42e9-baa4-6387be2fb68d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:48.764076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:48.764719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.764998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.765271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] PATH_INFO: `/v3/projects/7790bbe18b09475b81076e89944d2568` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.766416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] RBAC: Authorizing `identity:get_project(project_id=7790bbe18b09475b81076e89944d2568)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:48.767910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b36481c-798a-48d3-93c1-04da339caeb7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:48.769854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 204/416] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:48 2022] GET /identity/v3/projects/7790bbe18b09475b81076e89944d2568 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:48.775511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:48.814093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733a6f-5e6e-4494-b482-b80ca5bdf507 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.814605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733a6f-5e6e-4494-b482-b80ca5bdf507 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.815164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72733a6f-5e6e-4494-b482-b80ca5bdf507 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.816868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 205/417] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:48.831774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:48.832109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:48.832388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:48.866643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:48.867786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:48.924175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:48.929786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:48.949084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cc316-c52a-4996-9d72-a175952bfb02 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:48.975007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 206/418] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:48.976187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.003447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.005164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.005827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.006381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] PATH_INFO: `/v3/projects/7790bbe18b09475b81076e89944d2568/groups/3232bd8430f6458a8168f2fb87e9e2d8/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.013493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7790bbe18b09475b81076e89944d2568, group_id=3232bd8430f6458a8168f2fb87e9e2d8, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:49.015227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:49.039136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 213/419] 10.222.0.78 () {58 vars in 1442 bytes} [Wed Aug 24 09:46:48 2022] PUT /identity/v3/projects/7790bbe18b09475b81076e89944d2568/groups/3232bd8430f6458a8168f2fb87e9e2d8/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:46:49.114893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6961f14-39b5-4240-a656-7d7f8b03304a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.155663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.157230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.157786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.158342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.173360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:49.179518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:49.180540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:49.181367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:49.184149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 207/420] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:49.191932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.324475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a9ce0-6453-4f68-abb7-13270d87a9a0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.374779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.375683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.376511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.376931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.377342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.378418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:49.379770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:49.381338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:49.403502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 214/421] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:49.412228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.421933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:49.430845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:49.531703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890c5bb-6184-4875-8919-85644203f840 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.540374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ad0b1-bcf3-423b-9cb3-209bd43c0521 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.578180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.578827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.579106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.579378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.582690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.583544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.584097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.584586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.591680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:49.598621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:49.599527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:49.600368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:49.609634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 208/422] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:49.609634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.705843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:49.723945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea486f38-107a-4294-8a77-2c382c4dccd8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.760457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:49.762150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a435796-d3b8-49fc-b0c6-4472e8313c34 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:49.762534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.763149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.763433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.763704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.764619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:49.764918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:49.766302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:49.780897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 209/423] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:49.782414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.784142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:49.905989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0a9b3-c67f-4686-b0f2-da7dc4de4493 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:49.930844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 215/424] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:49.932875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cafbb0-e28a-4417-a735-3407c66f4b97 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.933303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cafbb0-e28a-4417-a735-3407c66f4b97 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.933583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cafbb0-e28a-4417-a735-3407c66f4b97 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.935652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 216/425] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:49.940235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cafbb0-e28a-4417-a735-3407c66f4b97 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:49.952923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:49.953756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:49.954043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:49.954319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:49.975321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.056714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.059148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.068573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cafbb0-e28a-4417-a735-3407c66f4b97 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.080101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.107562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.108366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.108809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.109290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.110662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.112302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3f02d-f56f-4e6b-89f1-2a8b65f656c6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.118842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 217/426] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.120056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.120449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.120856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.140980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:50.142365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:50.187894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 210/427] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:50.188780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.219113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 218/428] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:50.220168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.247433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.334183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3992eb-a4b6-4730-a4ff-e2bebf2d4b0f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.347705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.364902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175e6ee-b47c-4b0d-97b2-d80be88b5396 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.393608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.394274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.394558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.394837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] PATH_INFO: `/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.396492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=78e064f8e56e4c4a94dedba92fcf78aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.398446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9fc564-a0f3-473b-8a73-9b386b531cb9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.400685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 219/429] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/projects/78e064f8e56e4c4a94dedba92fcf78aa => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.404942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44d2-a0a0-446b-9658-4f3026eb372d None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.405416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44d2-a0a0-446b-9658-4f3026eb372d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.406088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc44d2-a0a0-446b-9658-4f3026eb372d None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.408121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 220/430] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:50.409061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc44d2-a0a0-446b-9658-4f3026eb372d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.424881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.425530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.425808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.426082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.427302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.428786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb5c25a-c3ab-4fe2-a47d-1826d46d0cdf None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.430904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 211/431] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.436363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.436363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.436363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.455949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:50.457154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:50.528448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 212/432] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:50.533859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.540642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc44d2-a0a0-446b-9658-4f3026eb372d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.579231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.580100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.580460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.580844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.582584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.583671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.585719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 221/433] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.586656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.596624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.640889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.662360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ddec6-b3e5-4498-9c34-9376dd798179 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.700478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.701183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.701525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.701886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.702658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:50.703118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.705436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.715102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c121198-d202-44e7-9446-771da3fe5c19 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.720187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 213/434] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=4156f70d4c1744029972dc81f38353a8 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.721551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.760334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.761359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.761844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.762345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.776355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:50.783252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:50.784244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 09:46:50.785046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:50.788440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 222/435] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:50.794663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.879015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a23efd-418e-4b56-8273-483b711c4673 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.892804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.919214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.919855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.920141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.920417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.921137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:50.921439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.922885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:50.930536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 214/436] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=78e064f8e56e4c4a94dedba92fcf78aa&role.id=00b090a41bf543a3a320c3787e8a836d => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:50.936723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:50.941631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb840a92-2af6-40d0-be1b-2ae463829a86 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:50.944043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:50.993302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:50.994108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:50.995037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:50.995495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:50.996656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] RBAC: Adding query filter params (name=member) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:50.997072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:50.999458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.020488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 223/437] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:51.020488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.134530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11cadc-901c-47ff-8fcb-da2cb4047f1a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.158328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc65fb63-b497-442e-9498-38ae88185964 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.196358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.197013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.197279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.197553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.209439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.210061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.210342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.210609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.217420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:51.223365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:51.224341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:51.225212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:51.227894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 215/438] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:51.231164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.254521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.275288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.312855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.313835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518fc186-599f-46e7-9ed0-05cbba63b0eb None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.358828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa391b-4f7b-42e9-b786-ee03a7bdc99c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.406743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.407357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.408454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.408454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.408707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:51.408910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.410284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.423979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 216/439] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:51.429047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.457140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 224/440] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:51.458091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a2be-9a83-4712-a8a3-ba81892a83fb None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.458379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a2be-9a83-4712-a8a3-ba81892a83fb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.458653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04a2be-9a83-4712-a8a3-ba81892a83fb None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.460044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 225/441] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:51.464900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a2be-9a83-4712-a8a3-ba81892a83fb None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.542113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.549275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87972ff0-2fea-4301-a9b4-104cbd0b52dd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.551696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.587324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.588005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.588290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.588560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.595351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04a2be-9a83-4712-a8a3-ba81892a83fb None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.629274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.629909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.630190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.630464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.631572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.633051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86ef6b-3797-46a9-9b2f-c136bbd8070a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.634696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 226/442] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:51.636964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.637246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.637516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.655110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:51.656620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:51.657049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.658463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.723976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 227/443] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:51.724820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.754152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.762225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 217/444] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:46:51.767850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.791171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:51.834895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31537b43-4ca7-4548-80bb-6f962ee5b8cf None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.871956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.872548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.872829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.873102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.874206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.875648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.877418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 228/445] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:51.878597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3f2113-10b6-4016-b636-a58f047f459e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:51.883130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.914107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:51.914665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:51.914944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:51.915215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] PATH_INFO: `/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:51.916353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] RBAC: Authorizing `identity:get_project(project_id=eb3b7ee937974dc182e3a2e76819d7c6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:51.917777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:51.919400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 218/446] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/projects/eb3b7ee937974dc182e3a2e76819d7c6 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:51.924718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:51.990737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.000103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e9d7e-e182-419c-ada1-e792be2c9550 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.029556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.030831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa39d7-7ef7-4ed8-af40-6d35d8d4ad17 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.035738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.036359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.036637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.036907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.037625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=4156f70d4c1744029972dc81f38353a8, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:52.037929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.039389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.045108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 229/447] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=4156f70d4c1744029972dc81f38353a8 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:52.045944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.078335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.079294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.079575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.079886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] PATH_INFO: `/v3/groups/420ad112404e4739b2440ef502b8d1e7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.081096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] RBAC: Authorizing `identity:get_group(group_id=420ad112404e4739b2440ef502b8d1e7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.083201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.084891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 219/448] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:51 2022] GET /identity/v3/groups/420ad112404e4739b2440ef502b8d1e7 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:52.090893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.170567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e3111-82be-4a16-b537-8a538ef7c6dd None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.206955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e7d5b6-788f-4fee-a3d8-4a6ed5b4605b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.210302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.210941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.215590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.215590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.225696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:52.231108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:52.232036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:52.233048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 230/449] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:52.237750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.246637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.252999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.254898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.254898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.254898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.255559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] RBAC: Adding query filter params (group.id=420ad112404e4739b2440ef502b8d1e7, role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:52.256040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.257668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.266613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 220/450] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/role_assignments?group.id=420ad112404e4739b2440ef502b8d1e7&scope.project.id=eb3b7ee937974dc182e3a2e76819d7c6&role.id=00b090a41bf543a3a320c3787e8a836d => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:52.309051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.364294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef242ff7-75d4-4495-bf14-a14e6affa977 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.401883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.402509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.402794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.403067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.403747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:52.404079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.405480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.414467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 231/451] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:46:52.420280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.525691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65262c-a098-4da5-b867-edbd47c1e50d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.563683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.565186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.565186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.565186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.627129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.635745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.637221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.734873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 221/452] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:46:52.741254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.838363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e533b4e-03d2-4dde-9f07-b0178611584c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:52.854604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.872893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:52.873507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:52.873785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:52.874054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] PATH_INFO: `/v3/projects/7790bbe18b09475b81076e89944d2568` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:52.875161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] RBAC: Authorizing `identity:get_project(project_id=7790bbe18b09475b81076e89944d2568)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:52.876615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:52.878223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 232/453] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/projects/7790bbe18b09475b81076e89944d2568 => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:52.884457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:52.973076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:52.980065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b754-bf83-4776-80e1-5b6f37114f1d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:53.013217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:53.013774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:53.014049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:53.014317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] PATH_INFO: `/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:53.015390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] RBAC: Authorizing `identity:get_group(group_id=3232bd8430f6458a8168f2fb87e9e2d8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:53.016860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042ffd9-ab95-4bdb-94fe-143cda23210c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:53.018395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 222/454] 10.222.0.78 () {56 vars in 1180 bytes} [Wed Aug 24 09:46:52 2022] GET /identity/v3/groups/3232bd8430f6458a8168f2fb87e9e2d8 => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:53.024136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:53.121089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0a308-0f79-4700-a06c-6b988ad5bfb9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:53.154351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:53.154950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:53.155231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:53.155504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:53.156255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] RBAC: Adding query filter params (group.id=3232bd8430f6458a8168f2fb87e9e2d8, role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=7790bbe18b09475b81076e89944d2568) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:53.156566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:53.157995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea15eae-0228-4b7a-a9b1-1c72962b516d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:53.164300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 233/455] 10.222.0.78 () {56 vars in 1376 bytes} [Wed Aug 24 09:46:53 2022] GET /identity/v3/role_assignments?group.id=3232bd8430f6458a8168f2fb87e9e2d8&scope.project.id=7790bbe18b09475b81076e89944d2568&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:55.449331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08cdd3-a4ea-4c82-b985-f809144c2de8 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.449845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08cdd3-a4ea-4c82-b985-f809144c2de8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.449947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08cdd3-a4ea-4c82-b985-f809144c2de8 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.451523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 223/456] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:55.457369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fdc03-4ffe-445b-b860-01f76f629112 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.457873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fdc03-4ffe-445b-b860-01f76f629112 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.457947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fdc03-4ffe-445b-b860-01f76f629112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.480924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-751fdc03-4ffe-445b-b860-01f76f629112 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:55.482490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-751fdc03-4ffe-445b-b860-01f76f629112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:55.551621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:55.585682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 234/457] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:55.759500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145279b9-3445-4184-90ee-3f9bbcd0b6ea None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.760200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145279b9-3445-4184-90ee-3f9bbcd0b6ea None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.760976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145279b9-3445-4184-90ee-3f9bbcd0b6ea None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.762765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 224/458] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:55.768225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772d2b-83a6-44da-97da-bd4c2f277381 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.768736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772d2b-83a6-44da-97da-bd4c2f277381 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.769252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772d2b-83a6-44da-97da-bd4c2f277381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.787664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c8772d2b-83a6-44da-97da-bd4c2f277381 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:55.789206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c8772d2b-83a6-44da-97da-bd4c2f277381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:55.876045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 235/459] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:55.927412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0ed7f-0c07-4165-a64b-4a9174d75287 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.928078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0ed7f-0c07-4165-a64b-4a9174d75287 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.928614 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac0ed7f-0c07-4165-a64b-4a9174d75287 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.930185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 225/460] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:55.945404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0ed7f-0c07-4165-a64b-4a9174d75287 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:55.945882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:55.946434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:55.946907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:55.968078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:55.970358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:56.106361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 236/461] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:56.132567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac0ed7f-0c07-4165-a64b-4a9174d75287 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.166927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.198173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.199843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.200351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.200831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.217871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.220794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.239131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:46:56.254130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:56.255084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:46:56.255980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:46:56.255980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.255980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:56.257096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 226/462] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:56.263263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.281784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.329812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04224b10-ab0b-4097-9af6-f1f3a8583a47 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.389529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.390503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.390984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.391473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.411185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.413635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:46:56.434017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:56.435213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:46:56.436114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:46:56.436114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.436114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:56.436114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 237/463] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:46:56.438420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.451219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292a665-3008-4387-820b-7ff2fae9eeef None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.490227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.490827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.491104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.491371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.492036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:56.492335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.493719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048711e-f956-46d7-be94-f3a4a94b1fc2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.513402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 227/464] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:56.514484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bd69-61d5-4028-8bbf-695bfaec93d9 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.514845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bd69-61d5-4028-8bbf-695bfaec93d9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.515193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236bd69-61d5-4028-8bbf-695bfaec93d9 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.520131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 228/465] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:56.520131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236bd69-61d5-4028-8bbf-695bfaec93d9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.549050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.560500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0bd244-d2b8-472d-b17a-6aaa79fda986 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.594958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.595590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.595907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.596180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.603065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.607717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.609195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:46:56.623556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:56.624513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:46:56.625385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:46:56.625385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:56.625385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:56.625385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 238/466] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:56.625680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.641136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236bd69-61d5-4028-8bbf-695bfaec93d9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.676833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.677437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.677714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.678081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.678609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:56.678907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.680311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06727e-481f-4a4a-900a-2fae523f4b4c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.697253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 229/467] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:56.698523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.698814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.699091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.718863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:56.720126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:56.743615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edac90-be2d-4050-97fd-4a75e0d56e97 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.781526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.782153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.782438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.782695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.783337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.784823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:56.795873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 230/468] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:56.795873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.817676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.828372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 239/469] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:56 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:56.829410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:56.877811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:56.923901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f8d020-3de2-4093-93d8-44a178290137 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:56.985363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:56.987778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:56.988309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:56.991459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:56.992395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:56.992847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:56.994419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.019935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 231/470] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:46:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:57.026293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:57.053808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb020cdf-a5ae-4aac-b8ad-66bbdcb38315 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.100388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.101252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.101532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.102281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.103017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.104519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.130031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 240/471] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:56 2022] POST /identity/v3/users => generated 248 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:57.131529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:57.179500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468da2a-27eb-4bd0-b1dc-074534f39380 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.210260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:57.211000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:57.228471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.229440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.229780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.230063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.231769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.233599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9a30-9d39-486a-9f27-1d1e16845851 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.268239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 232/472] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:57 2022] POST /identity/v3/users => generated 248 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:46:57.270335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb4a1-e71a-41f1-84f4-ec0c553838c2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.270634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb4a1-e71a-41f1-84f4-ec0c553838c2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.270911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb4a1-e71a-41f1-84f4-ec0c553838c2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.272609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 233/473] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:57.273226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28885f-f5f2-4dca-a83c-9a084b2e9a49 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.278275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.278578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.278843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.317019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:57.318287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:57.322493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.323445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.323985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.324513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.337160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.339064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:46:57.355745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:57.356953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:46:57.358606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:46:57.358606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:46:57.358606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.360576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 241/474] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:57.366618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:57.407894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 234/475] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:57.493821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcd851-336c-472e-8aab-f42d75d17537 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.501977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:57.539157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.540140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.540617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.541070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.541726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:57.542029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.543421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.557481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 242/476] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:57.564184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:57.699106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:57.715896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba75a8-9e31-47f8-9061-0d8e05048a6c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.740203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:57.802745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.803380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.803653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.804004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.804674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.806156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.839353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188e03-4f85-4f70-b82e-9d446e9a317a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:57.840285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 235/477] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:46:57 2022] POST /identity/v3/users => generated 249 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:46:57.880478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:57.881168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:57.881456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:57.881733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:57.882426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:57.883849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:57.893614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.sql.core [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:46:57.893614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:46:57.893614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [parameters: {'id': '3aef40f463ec462799700693c1e1d050', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:46:57.893614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result.read() Aug 24 09:46:57.912231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:46:57.913186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application yield resource Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise self.value Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application session.commit() Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:46:57.914102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:46:57.914953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:46:57.915950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result.read() Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:46:57.918588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [parameters: {'id': '3aef40f463ec462799700693c1e1d050', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:57.925843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 09:46:57.927757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:46:57.929738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 243/478] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 24 09:46:57 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:57.929871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:58.039596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69402f1a-7fb3-4eb4-a9cf-bcb0516a9351 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:58.076636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:58.077543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:58.077774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:58.077926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:58.088749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-090a874e-c3a2-44c7-837d-92f46bd774cd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:58.094305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:58.096416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 09:46:58.098327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:58.099761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 236/479] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:46:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:58.103161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:58.142077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:58.217685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc1d871-13d8-4590-b5bd-290526c5e68d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:58.256170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:58.256781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:58.257064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:58.257336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:58.258005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:46:58.258309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:46:58.259690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3880fd15-7d77-4cdc-a8b3-e15c293ca357 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:46:58.272808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 244/480] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:46:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:46:59.300816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e37a5-08d5-424b-963f-9e71b3f292ab None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.301337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e37a5-08d5-424b-963f-9e71b3f292ab None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.301337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0e37a5-08d5-424b-963f-9e71b3f292ab None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.303081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 237/481] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:59.307963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bea-48b0-4d41-9834-0ff26588a628 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.308367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bea-48b0-4d41-9834-0ff26588a628 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.308556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d617bea-48b0-4d41-9834-0ff26588a628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.332288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6d617bea-48b0-4d41-9834-0ff26588a628 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:59.333589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6d617bea-48b0-4d41-9834-0ff26588a628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:59.409931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 245/482] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:46:59.710129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e37a5-08d5-424b-963f-9e71b3f292ab None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:46:59.730042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c608-8bec-4025-9776-4d73d8ccd0bb None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.730375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c608-8bec-4025-9776-4d73d8ccd0bb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.730607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668c608-8bec-4025-9776-4d73d8ccd0bb None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.732349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 246/483] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:46:59.737705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1d4bd-be81-4127-bf8e-04747a1ccf39 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.737993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1d4bd-be81-4127-bf8e-04747a1ccf39 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.738267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1d4bd-be81-4127-bf8e-04747a1ccf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.761476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:59.768785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1bd1d4bd-be81-4127-bf8e-04747a1ccf39 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:59.770273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1bd1d4bd-be81-4127-bf8e-04747a1ccf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:59.789851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:46:59.887656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 247/484] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:46:59.898101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2972-4ef9-4f50-bca4-c4717f2c1245 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.898428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2972-4ef9-4f50-bca4-c4717f2c1245 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.898656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee2972-4ef9-4f50-bca4-c4717f2c1245 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.900652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0e37a5-08d5-424b-963f-9e71b3f292ab None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:46:59.903484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 248/485] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:46:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:46:59.908072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.908360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.909195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.940300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:46:59.941525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:46:59.956907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:46:59.957549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:46:59.957830 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:46:59.958106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:46:59.969161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:46:59.975496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:46:59.976518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:46:59.977519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 238/486] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:46:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:46:59.983418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.092969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 249/487] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:00.113462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.131167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f594c47-808a-45ee-ab02-aac0a381d6a0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.170076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.170682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.170964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.171238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.171923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:00.172227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:00.173753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:00.182532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 239/488] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:00.189171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.267656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.305115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.341703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a6dcb-66d6-4d44-ab0b-e973e531abab None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.424424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d4fc96-1007-4d00-8f2e-e956613b31ce None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.427998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.428641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.428920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.429193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.452094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:00.453588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:00.468750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.469386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.469682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.470145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-40d01b8a-75f4-4387-a1dd-5381c591de07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:00.477162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:00.478254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:00.479328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:00.479328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:00.479328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:00.481139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 250/489] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:00.482418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:00.482750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e9fe-ce0e-4025-8d1c-817ba4261843 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.483173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e9fe-ce0e-4025-8d1c-817ba4261843 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.483585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2e9fe-ce0e-4025-8d1c-817ba4261843 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.485288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 251/490] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:00.486620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e9fe-ce0e-4025-8d1c-817ba4261843 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-51b365a2-216e-4ec8-8008-e8e17bf5ab94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:00.492546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:00.493674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:00.494617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 240/491] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:00.495271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.495442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.495716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.516854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:00.518332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:00.521332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.601899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2e9fe-ce0e-4025-8d1c-817ba4261843 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.602097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 241/492] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:00.608675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.640954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.641572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.641854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.642128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.642774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:00.643080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:00.644886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:00.646562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.663537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 252/493] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:00.663537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.725444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b2c7e-d9b0-4ead-8d01-2459c6b6e52b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.763394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.764334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.764851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.765357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.766262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:00.766816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:00.768494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:00.773539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d763f36-52dd-4c94-8168-85acdd4f7e77 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.774691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.782767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 242/494] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:00.783733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.817116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.818488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.819017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.819438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.832372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:00.838422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:00.839494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:00.840401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:00.841670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 253/495] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:00.841836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.893178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae0523-66c9-4648-bfd3-e618b98a42e2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.907230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:00.927488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.928183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.928459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.928727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.936031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:00.937484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-4fc93b08-5d87-436c-b16a-a4ca593f8fba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:00.949735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:00.950654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 243/496] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:00.957573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a202c5c-1183-411d-a4c6-f00e2d414e0a None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.959073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891eaca1-b5f4-455d-aff1-c9241a1742c6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:00.959164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a202c5c-1183-411d-a4c6-f00e2d414e0a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.959164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a202c5c-1183-411d-a4c6-f00e2d414e0a None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:00.959164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 244/497] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:00.959164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a202c5c-1183-411d-a4c6-f00e2d414e0a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:00.992876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:00.993764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:00.994045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:00.994318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.004309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.005766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.015658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:01.021168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:01.022094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:01.022960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:01.022960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.022960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:01.022960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 254/498] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:01.022960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.061487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a202c5c-1183-411d-a4c6-f00e2d414e0a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.094730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.095330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.095609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.095909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.096571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:01.096871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.098913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.106752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 245/499] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:01.107531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.121771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069fabf3-b24a-4ddf-968b-190478ec4f94 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.128752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.158079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.158700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.158978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.159246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.169385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f37ae90e-aa84-4f2d-8dc0-0d23a6591c79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:01.174145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:01.175134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:01.176683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 255/500] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:01.177878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.177878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.177878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.195422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:01.196670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:01.210766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd38a82-68e8-4b7f-9134-258ddecb3d5a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.232431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.245161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.245763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.246042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.246319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.247041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:01.247344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.248807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.262610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 256/501] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:01.263450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.287133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 246/502] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:01.287927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.382192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.390179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a9beb-24b2-4722-bd54-51385a961e33 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.393982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47736b26-12a1-4cc0-b61b-b262804bd21a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.430487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.431091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.431372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.431646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.432243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.432702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:01.432702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.433675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.434133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.434234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.434677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.449172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.454677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 257/503] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:01.454803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.454879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:01.467740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:01.468744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:01.469599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:01.469599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.469599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:01.473645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 247/504] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:01.474487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.507468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.565590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf23d8-9709-48c5-a018-065e2399da3e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.579599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58de154-c6a4-4182-9e4a-58175046f88d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.599210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.599852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.600129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.600407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.601068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:01.601371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.602746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.610545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 258/505] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:01.611313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.615449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.616388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.616663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.616928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.630480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.685215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.686673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.710944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54608f0c-da6b-4983-b3ea-420fbdac8e70 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.717372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.743983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.744601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.744881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.745145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.752506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.753930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:01.766659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:01.767902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 259/506] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:01.768799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.793412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 248/507] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:01.794371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.849051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.872385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20d462-4eeb-4ec0-b805-c6c5ba2e2131 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.895400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfee92e2-e775-40fd-b936-a2c5e49f2ec9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:01.906372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.906743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.907024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.907302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.907975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:01.908282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.909709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.921462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 260/508] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:01.922443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:01.929843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:01.935313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:01.935976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:01.936249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:01.936518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:01.946989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:01.948453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:01.963041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:01.964144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:01.965105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:01.965105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:01.965105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:01.965105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 249/509] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:01.965376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.025952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631afe0-2a74-481a-ac08-3b081cbb4acf None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.063803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.064544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.064892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.065233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.065323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd61807-4502-4d1d-96b7-a2beefa92a9d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.072310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.076678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.078143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:02.093204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:02.094271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:02.095314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:02.095314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:02.095314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:02.095314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 261/510] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:02.095571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.102488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.103041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.103319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.103588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.104350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:02.104657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.106496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.124816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 250/511] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:02.125610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.173333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.200255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6771f2-0e55-4bb7-bb49-7e331da7e4db None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.233230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.233703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15391e8b-9ba5-4cc2-9ccc-19daa1271ce3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.233853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.234135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.234410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.245323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.246785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:02.260864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:02.261925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:02.262784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:02.262784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:02.262784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:02.262784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 262/512] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:02.262784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.271297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.272938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.273333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.273733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.281515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.283149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:02.294523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:02.295586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 251/513] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:02.296870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.308881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.366052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fddb05d-6e9f-41b6-bf8a-e34240deeb64 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.399102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ecc315-0061-4503-8f5f-9c980546edfb None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.399499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.400131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.400407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.400680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.400787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.401317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:02.401632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.403156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.415001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 263/514] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:02.415747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.438736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.439340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.439621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.439924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.440564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:02.440865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.442237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.454381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 252/515] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:02.455980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.518407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70705fb3-c643-4aac-aa77-28ffb394fc32 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.520156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.554433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.555036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.555319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.555594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.558457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881de75-4e9c-4d56-8883-7475a09c9847 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.592219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.592822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.593100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.593374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.594011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:02.594310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.595687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.607517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 253/516] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:02.608334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.626728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.628269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.632466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.706365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c107e38-1b78-4521-8217-f40c785708df None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.727182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 264/517] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:02.728029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.730231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.740972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.742181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.742462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.742734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.743465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:02.743771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.745216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.762117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 254/518] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:02.762905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.829072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ee598-b705-413f-a0c3-6f62bdc28bc5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.844932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:02.863769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.864404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.864609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d9b659-fb14-48ca-a299-bd39bb0f18f8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:02.864734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.864946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.865584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.867648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.888614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 265/519] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:47:02 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:02.889363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.901589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:02.902244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:02.902526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:02.902797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:02.911173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:02.913154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:02.926048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:02.927110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 255/520] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:02.928042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:02.973502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.029482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f220-921c-4d53-897e-6d323eb76571 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.053857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baf663-cf2f-4c00-9510-b831ac9ca08a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.078712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.079355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.079632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.079937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.088183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.089647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.092349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.093281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.093567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.093846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:03.103240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:03.104166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 266/521] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:03.105440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.106877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.108903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.110592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:03.130065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:03.131000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:03.132663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:03.132663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:03.132663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:03.132663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 256/522] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:03.132663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.220543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c2fb3-a568-40cf-b86f-d2b8fff5c6fe None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.237709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.247083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988cd77-5821-4ed5-b93a-51795e29d1b8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.255723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.256356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.256638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.256913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.284487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.285422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.286050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.286464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.287387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:03.287965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.290085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.309733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 257/523] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:03.310741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.332335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.334070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.364642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.417178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40bcf52-0cfa-47fb-83aa-4d13ac548090 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.446693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 267/524] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:03.446693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.463987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.464689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.464968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.465246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.465951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:03.466260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.467701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.472004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.487155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 258/525] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:03.488040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.569556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d034e-7d3a-4df4-8df2-c6d4db875ab3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.598742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd37e5-e168-4ef6-b773-eb50b38bc49c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.608690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.609891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.610353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.610765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.611195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.612180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:03.612668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.614184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.630503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 268/526] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:03.630503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.643083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d178-df25-4cfa-945d-11617637171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.643695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d178-df25-4cfa-945d-11617637171a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.644006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d178-df25-4cfa-945d-11617637171a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.644281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d178-df25-4cfa-945d-11617637171a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.712194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.720482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d178-df25-4cfa-945d-11617637171a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.721966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d178-df25-4cfa-945d-11617637171a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.735280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f010b8-d144-4c05-adc8-dd18bf2a1320 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.769061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.769705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.769987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.770262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.781243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.782701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:03.797530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:03.798664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:03.799538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:03.799538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:03.799538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:03.799538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 269/527] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:03.799538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.834561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 259/528] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:03.835708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d178-df25-4cfa-945d-11617637171a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:03.849067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.916403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d75e86-2bbc-4dfd-b78c-1b92b1efc33a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.945335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:03.949678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d178-df25-4cfa-945d-11617637171a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:03.951774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:03.952525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:03.952908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:03.953287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:03.954130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:03.954547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:03.956169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:03.963111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 270/529] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:03.964040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.000529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.001245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.001808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.003437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.013034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.014897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:04.030840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:04.031966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.033035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:04.039314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 260/530] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:04.039314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.113671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc254b-ee9b-47a2-aa4a-a0cce148a4d4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.118008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.162421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.163058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.163343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.163622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.164318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:04.164495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0c80b4-f8c5-4237-ace6-8579ecccd626 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.164614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.166677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.183927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 271/531] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:04.185130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.204472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.205170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.205451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.205767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.220733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.222972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:04.242981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:04.244082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:04.244962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:04.244962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:04.244962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:04.249753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 261/532] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:04.249753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.268086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.318930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a847001a-8b6b-4f9c-a870-ad9cf0616f8d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.360825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.361926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.362732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.363184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.364076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:04.364526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.366929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.381488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecaea-3dc2-465d-841d-998bf724d557 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.402177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 272/533] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:04.402177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.417392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.432701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.433789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.434305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.434685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.442521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.444146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:04.465941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:04.467282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.468175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:04.472146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 262/534] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:04.473011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.541802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ed4a04-f904-4a53-82b4-3b145a2bc94e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.561677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.586052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce023242-24d6-45f9-9335-6af7a639fe98 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.592054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.592745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.593098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.593439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.594180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:04.595150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.596996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.613287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 273/535] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:04.616555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.633138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.634214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.634551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.634944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.636115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=3061244943c84217bf9ba30224cd4951) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:04.636431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.638536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.647043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 263/536] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/role_assignments?user.id=3061244943c84217bf9ba30224cd4951&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:04.648343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.716934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.743047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4587522a-15d7-4a31-be60-0baaaa39e086 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.785314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e6f80-2177-4b93-834b-df8080cb75f5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:04.789975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.790592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.790877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.792072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.793044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:04.793408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.794954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.814483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 274/537] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:04.815893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.840350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:04.841005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.841289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.841562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.855642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:04.858136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-4f563735-9467-49fe-825d-93cc592792d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:04.877409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:04.879413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:04.881615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:04.883329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 264/538] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:04.887856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:04.890237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d625a0-7e4f-4248-8d29-1fa919effdee None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:04.890517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d625a0-7e4f-4248-8d29-1fa919effdee None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:04.890804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d625a0-7e4f-4248-8d29-1fa919effdee None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:04.892367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 265/539] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:04.895831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d625a0-7e4f-4248-8d29-1fa919effdee None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:04.981253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb440cb-b5b9-4be6-9d55-05edb4e7d68f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:05.012064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d625a0-7e4f-4248-8d29-1fa919effdee None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:05.020356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:05.031627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:05.032742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.033549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.033969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.034847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=6b171220975d48488be864348cc9aca8) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:05.035303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:05.037504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1248336-eb40-4f5e-89e0-b7842d19a2cc None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:05.046447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 275/540] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/role_assignments?user.id=6b171220975d48488be864348cc9aca8&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:05.048696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.049242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.049780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.053095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:05.053926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.054595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.055174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.056719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:05.058084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:05.061332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:05.072213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:05.074419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:05.082612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 266/541] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:05.095837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:05.174211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 276/542] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:05.260967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78033678-a812-4a4d-9019-6dd24537759f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:05.304615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:05.305160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.305438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.305704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.306433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=77e0d9362b3241d3b888e1cb0a2b5f4d) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:05.306740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:05.308851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:05.317164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 267/543] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:05 2022] GET /identity/v3/role_assignments?user.id=77e0d9362b3241d3b888e1cb0a2b5f4d&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:05.550977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:05.580355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:05.658736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbbbf5-03bd-4914-8bec-13e00afb3b73 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:05.700280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:05.701279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.701759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.702232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.713329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:05.719463 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:05.720364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:05.721168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:05.723718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 277/544] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:05.729849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:05.769772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:05.874742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45b220-bf7b-4398-bd5b-170db3745c76 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:05.918905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:05.919949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:05.920435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:05.920909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:05.921868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:05.922382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:05.924068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:05.940678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 268/545] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:05.946428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.041536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:06.054273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256ff6f-3b3c-499b-9175-395b5328229f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:06.090226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:06.090855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.091134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.091409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.101658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:06.103129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7ca028b3-ca6e-4622-a208-7745aef0acca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:06.118380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:06.119357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:06.120245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:06.120245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:06.120245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:06.120245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 278/546] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:06.127244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.255195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:06.266027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fd21c-41de-4e01-b094-6ca7540c5439 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:06.301232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd69896-c0e9-474e-8cb8-b421fabd1871 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.301529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd69896-c0e9-474e-8cb8-b421fabd1871 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.301783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd69896-c0e9-474e-8cb8-b421fabd1871 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.303376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 279/547] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:06.306806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:06.307422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.307704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.307999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.308433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab25-f588-4975-a086-018f2208a27f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.309158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:06.309267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab25-f588-4975-a086-018f2208a27f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.309515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083ab25-f588-4975-a086-018f2208a27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.309622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:06.310972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:06.323852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 269/548] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:06.329850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.347686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e083ab25-f588-4975-a086-018f2208a27f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:06.348952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e083ab25-f588-4975-a086-018f2208a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:06.487746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f4c0f-9827-403d-acf5-a35844c67acd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:06.494573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 280/549] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:06.528550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:06.529267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.529584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.529859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.537281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:06.538751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:06.550576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:06.552169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:06.553796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 270/550] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:06.557473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ab25-f588-4975-a086-018f2208a27f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.667208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:06.702499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083ab25-f588-4975-a086-018f2208a27f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:06.738961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:06.739945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.740452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.740925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.741901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:06.742412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:06.744069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:06.783580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 281/551] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:06.791915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.796063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:06.826181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:06.920514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4265264-8550-4bc6-982a-7ba654f7bbf3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:06.948149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:06.957276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:06.960029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:06.960029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:06.960029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:06.970972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:06.975943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:06.976841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:06.977830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:06.978474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 282/552] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:06.983916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.034811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6fe7b1-e002-4104-80ca-e019b64c9dbd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.073829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.074448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.074726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.075004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.107531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a797787-1e5d-4f11-824a-64f568d620e5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.145686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.146364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.146643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.146918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.147688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:07.148033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.149549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c279d3-8365-4c99-8133-4b3e5b33100d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.151428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.153139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da184a3-ee35-4e69-ada7-1e24ff5973e4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.163283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 283/553] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:07.164743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc627-9a01-48bf-a933-46763029f616 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.165034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc627-9a01-48bf-a933-46763029f616 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.165308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedc627-9a01-48bf-a933-46763029f616 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.166911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 284/554] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:07.167455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc627-9a01-48bf-a933-46763029f616 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.172152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.225271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.274591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 271/555] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 09:47:07.276579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.277071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.277584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.280878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedc627-9a01-48bf-a933-46763029f616 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.299098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:07.300724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:07.316433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.317073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.317352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.317682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.329471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.330930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:07.348447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:07.349441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:07.350343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:07.350343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:07.350343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:07.354990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 285/556] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:07.354990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.384196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 272/557] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:07.385320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.469775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.472611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f429771-4bd8-4915-bb74-9b77528ccb1b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.507312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.511536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.512269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.512622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.512962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.525624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.527705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:07.544964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:07.546582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:07.547683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:07.547683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:07.547683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:07.548178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 286/558] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:07.554719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.601345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30af3f8-d49c-422a-b02a-616bea894af6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.644295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.645001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.645351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.645698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.646426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:07.646801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.648284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a93c6b-97c3-48e0-93d1-1f9dcec83a7f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.660962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 273/559] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:07.662297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f014-9a21-450a-a5b7-488b78765f0d None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.666928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f014-9a21-450a-a5b7-488b78765f0d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.667291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f014-9a21-450a-a5b7-488b78765f0d None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.668918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 274/560] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:07.669792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f014-9a21-450a-a5b7-488b78765f0d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.679832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43a811-0192-417e-943d-a0967a72a3c8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.719279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.720409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.720698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.721024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.721959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:07.722274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.724418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609e8dd2-3d04-423f-9572-fcae6fe5cd08 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.741214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 287/561] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:07.742468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.742757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.743017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.766770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:07.768009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:07.777825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.786582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f014-9a21-450a-a5b7-488b78765f0d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.788294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:07.824689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.825312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.825517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.825788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.833335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:07.834890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:07.838707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 288/562] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:07.839549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:07.846404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:07.847368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 275/563] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:07.849008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:07.954457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bc7f7-70a1-418a-989d-f4ef6a6c9e8b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.954768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b2eb-5f39-40f2-a580-715bcf26352b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:07.990712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.991599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.992080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.992563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:07.993834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:07.994469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:07.994750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:07.995022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.002397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.003868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.004758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-95ece1d0-9009-4ee5-a9a9-d36ca5f47595 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:08.010238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.011159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.012022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:08.013931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 276/564] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:08.015625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.016489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:08.017374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 289/565] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.017863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.018183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f161-072b-4c48-9eae-03d75a7256b9 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.018595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f161-072b-4c48-9eae-03d75a7256b9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.018866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7395f161-072b-4c48-9eae-03d75a7256b9 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.022096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 277/566] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:08.022096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f161-072b-4c48-9eae-03d75a7256b9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.030265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.048925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.132812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7395f161-072b-4c48-9eae-03d75a7256b9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.133581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf53b0f0-de2c-47dc-bc8d-ae5f32f4685f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.165471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.166072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.166350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.166620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.167350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:08.167601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.168426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.169038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.169129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.169321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.169596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.170326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:08.171174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.172706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821c3842-a7e4-4611-ac76-9ee3240d5c52 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.181241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 278/567] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:08.182028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.195470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 290/568] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:08.196799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.197092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.197372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.216582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:08.218618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:08.270261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.274596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.286388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 291/569] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:08.287194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.294512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168f44a-c564-4825-8b09-4ded82db3e21 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.331947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.332868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.333315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.333594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.334265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:08.334573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.335968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.344469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 279/570] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:08.345258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.399275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60991c3a-98d9-4b78-a906-1b978f0ce8e1 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.443186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.444049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.444660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.444948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.451551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401ee7-606c-43fe-a66d-59cc0ff86c06 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.460635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:08.466152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.467124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 292/571] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.468074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.494939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.495847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.496315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.496806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.497609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=61f9de1f0b754d74bf44b92eb2134081) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:08.498018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.500449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.515834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 280/572] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/role_assignments?user.id=61f9de1f0b754d74bf44b92eb2134081&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:08.515834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.529228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.540958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.598745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9d7d39-ce50-4af8-8006-0709daa0ad83 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.637571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.638222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.638506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.638780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.639209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e8161-b0b4-4688-975b-46c98cce3755 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.652212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.653637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:08.670167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.672141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:08.673303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:08.673303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:08.673303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:08.673303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 293/573] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.673303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.685522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.686332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.686608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.687593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.699053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.700516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:08.716815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.717786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:08.719713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:08.719713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:08.719713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:08.719713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 281/574] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.719713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.783590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.788484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316cedf-d283-4989-ad7e-620a97dc5f26 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.790922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:08.827483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.828585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.829141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.829422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.830099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:08.830408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:08.830709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81de772-3a33-4a7a-89f7-c56fc66d97cd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.831820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:08.847108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 294/575] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:08.847108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.865596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:08.866237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:08.866518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:08.866792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:08.877855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:08.882974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:08.883930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 282/576] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:08.884801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:08.965867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00364ff9-e881-45aa-b574-9a024db10bf3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:08.996603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123e05-6ffd-43b8-bb10-83332c771cb4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.001821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.002433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.002712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.002985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.003633 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.003970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.005340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.021820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 295/577] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.021820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.022073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.039962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.040712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.040993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.041272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.041920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.042223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.043626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.051448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.056949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 283/578] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.057780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.135610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84240f89-ebc5-44d3-80a9-55e5dace580e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.167009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29411-0f98-4ea4-ac26-34068673aac0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.173260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.173958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.174244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.174516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.185549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.187013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.202132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.202744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:09.202845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:09.208801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:09.209847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:09.209847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:09.209847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:09.210017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.210017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.210017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.210211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 296/579] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:09.210281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.210349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.210416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.217843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 284/580] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.218555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.265124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.300703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.323104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b2db-9c01-4741-9b5d-7c958f6b38fa None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.329231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b351de-b612-4d19-bfd3-98374b70bdce None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.358598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.359062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.359341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.359735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.363718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.364376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.364654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.364924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.366699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.368176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.372739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.374269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:09.380279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:09.381157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 297/581] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:09.382133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:09.390615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:09.392286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:09.393243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:09.398985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 285/582] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:09.398985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.500382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0d92fe-b1c9-47bf-91f8-75c3c14849b5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.510337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd9c78-2415-41b8-bce2-048f8617bc0b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.511980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.540535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.541451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.541883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.542166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.542828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.543122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.544529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.549656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.550280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.550557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.550827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.553082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.558207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 298/583] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.559038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.566603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.568088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:09.583088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:09.584125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:09.585004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:09.585004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:09.585004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:09.585004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 286/584] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:09.585871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.678311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aae08d-164b-4317-bae2-37478ee4bd9e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.691741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611fc45-9535-4efb-8f1f-7c814cae5be3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.715573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.716213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.716502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.716783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.717479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.717777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.719220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.730765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.731496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 299/585] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.731907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.732332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.732582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.733083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.733969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.734424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.737258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.759215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 287/586] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.760671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.778292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.795352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:09.841734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc632e74-977e-4921-a6b5-560341fd5748 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.872548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74d7c8-ce9c-4c1e-b367-ad20d29e3edb None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:09.879214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.879884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.880173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.880451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.888570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.890021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:09.901450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:09.902406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 300/587] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:09.903414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:09.913675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:09.914290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:09.914562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:09.914834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:09.915474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:09.915771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:09.917150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:09.930570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 288/588] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:09.931443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.007481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.022367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487edc31-e596-4c41-98b7-c8b6e6f85736 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.041434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.041645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b78ba-add6-4f91-909a-079f5bbc26f9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.060478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.061141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.061428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.061705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.077304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=7f645f811e804835a793f372a1ac7111, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.078756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.079749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.080412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.080695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.081009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.092499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.104206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 301/589] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:09 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:10.104440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.104541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:10.121737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:10.122945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:10.123888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:10.123888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:10.123888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:10.123888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 289/590] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:10.125009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.250026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ef8ae-7cd6-450e-aa5c-0180ba184349 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.271808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ee916-7968-479a-8ed4-3a2b495676d5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.302781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.303199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.303410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.303650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.303955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.304699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:10.305006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.306444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.316283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.317693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.317693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.317693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.325749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.326320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 302/591] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:10.327287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.327397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:10.339286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:10.340276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:10.341284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 290/592] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:10.341874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.365743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.469381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ad8410-afe5-4294-b664-a344780e26c3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.486450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a781215-d958-404e-a22f-de0227cb2845 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.508969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.509823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.510105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.510382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.511033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:10.511340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.512798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.524316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.524992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.529681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 303/593] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:10.529799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.529799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.529955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.543019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.544513 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c1842834-cbd5-4b8c-b6f7-4ff3b3a913a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:10.559964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:10.560988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:10.561877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:10.561877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:10.561877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:10.561877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 291/594] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:10.562752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c476a2-c2ed-47de-937e-e9bd1db2f3fe None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.563105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c476a2-c2ed-47de-937e-e9bd1db2f3fe None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.563451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c476a2-c2ed-47de-937e-e9bd1db2f3fe None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.567825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 292/595] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:10.567825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c476a2-c2ed-47de-937e-e9bd1db2f3fe None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.606473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.630909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.648903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f0c584-3429-42d6-b3ec-59b4e1feaf0c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.675995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c476a2-c2ed-47de-937e-e9bd1db2f3fe None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.684806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.685416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.685702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.685978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.686835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:10.687150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.688614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.707248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 304/596] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:10.708260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.720067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.720835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.721265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.721543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.732885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.734538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9508037b-8cd6-48bc-9b2d-962652893c8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:10.747347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:10.748329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:10.749231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 293/597] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:10.750361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.750361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.750361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.770205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:10.771395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:10.826967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a38761-4dbb-487b-9c65-92a243945682 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:10.843832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 294/598] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:10.843832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.874741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:10.875618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:10.875931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:10.876208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:10.876854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:10.877192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:10.878693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:10.881641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.892317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 305/599] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:10.893187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:10.896730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:10.973799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d4858-e802-4f5e-beda-ff714162b57b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.004412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6aac68-c0a4-43aa-9068-c31da1369664 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.019843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.021188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.021882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.022335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.036794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.039146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.042234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.042919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.043276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.043621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.044420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:11.044794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.046354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:11.058406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:11.059406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:11.060353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:11.060353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:11.060353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:11.067312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 306/600] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:11.067312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.067839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 295/601] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:11.068837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.156688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.171823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.196481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955c993-d6fb-4469-b5fc-126ad02a4970 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.199623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42b0b8c-a6f4-461b-be18-1ab8901d8de0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.233270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.233932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.234213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.234313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.234503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/3061244943c84217bf9ba30224cd4951/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.234941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.235220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.235492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.243068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.244548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.250016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=3061244943c84217bf9ba30224cd4951, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.251473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:11.256491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:11.257634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:11.258914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 307/602] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:11.259413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.270860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 296/603] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:11 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/3061244943c84217bf9ba30224cd4951/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:11.272888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.392658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70363350-ea8a-40c3-bb03-81a1e05c6057 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.398850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700d8ee-3b04-4e16-93af-6fa112a8a9fd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.423231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.431069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.443827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.444455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.444747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.445145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.446148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:11.446460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.447906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.450662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.452681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.453180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.453665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.469032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 308/604] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:11.469179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:11.469260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:11.475058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:11.476109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:11.476947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 297/605] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:11.477647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.584006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc6ae4-9bc2-4aee-a4e3-b0449e428004 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.626475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949f708-df70-41c2-9791-8e7745ecd5fe None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.628861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.629827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.630329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.630802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.631693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:11.632222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.633878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.649704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 309/606] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:11.650952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.671618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.672342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.673007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.673345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.682754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.685208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:11.701328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:11.702283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 298/607] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:11.703785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.707825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.769196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18298a70-2a8c-4911-afa8-08fa1e81d7d5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.769787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:11.808820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.809795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.810407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.810883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.811746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:11.812271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.813842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.826549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 310/608] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:11.827675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.846165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ade8af-33a9-4255-8fb7-7520514b2b47 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.892521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:11.893190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:11.893579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:11.894451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/6b171220975d48488be864348cc9aca8/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:11.917995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=6b171220975d48488be864348cc9aca8, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:11.919971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:11.938525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 299/609] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:11 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/6b171220975d48488be864348cc9aca8/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:11.939904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:11.961852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcc56e-2832-4e11-82ae-cecc98332c39 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:11.978073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:12.055248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.055908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.056198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.056503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.057171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:12.057478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.058952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.072905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:12.078140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90bd93-f5b9-4361-a1d1-7d002e1a0af7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:12.088252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 311/610] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:12.094131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.123642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.124952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.125830 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.126309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.141362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.143405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:12.163920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:12.164953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:12.165867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:12.165867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:12.165867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:12.168639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 300/611] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:12.174202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.220083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4162c-5944-4afa-9431-a4b606b56bac None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:12.271005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.271679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.271995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.272272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/77e0d9362b3241d3b888e1cb0a2b5f4d/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.302103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=77e0d9362b3241d3b888e1cb0a2b5f4d, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.303571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47c32c-6b3d-4a63-b5ac-4ae254eaa87f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.319060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a9827-4b4b-4b16-addb-94fe36ec2fdd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:12.327718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 312/612] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:12 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/77e0d9362b3241d3b888e1cb0a2b5f4d/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:47:12.328994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09361-60e0-4d4a-a933-19a6e0300825 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.329286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09361-60e0-4d4a-a933-19a6e0300825 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.329563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e09361-60e0-4d4a-a933-19a6e0300825 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.333966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 313/613] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:12.350114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.350424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.350666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.372266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:12.373822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:12.390055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:12.402701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.403367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.403651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.403958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.404625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:12.404932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.406474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.432189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 301/614] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:12.441859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.472285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:12.514074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 314/615] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:47:12.601351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22368c-2050-440a-b3a9-e4a542b96280 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:12.657079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.657955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.658376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.658789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.668278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.669933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:12.686460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:12.687612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:12.688773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:12.690521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 302/616] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:12.697028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.826515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653b2740-178c-45f7-8229-40ff8d6596d4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:12.839741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.875392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:12.876722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:12.877159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:12.877592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:12.878457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:12.878910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:12.881651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:12.922928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 315/617] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:12.938189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:12.942127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:12.960167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:13.021979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7541b-b27d-4240-b894-409f8cad0c59 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.064965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.065966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.066459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.066921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.092723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:13.102933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:13.103958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:13.104803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:13.107950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2884e4d-c437-431c-ba32-f1b146a04d32 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.117394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 303/618] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:13.118591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.171396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.172496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.172986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.173457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.192904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.194781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:13.221614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:13.222612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:13.224336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:13.224336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:13.224336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:13.230481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 316/619] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:13.234670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.319837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa222a-8c9a-4245-bef6-d963848da24b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.345040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:13.386890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:13.399759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.400717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.401195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.401696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.403612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:13.404164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.408876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.424969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 304/620] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:13.430717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.437811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c2061-e8cd-405f-bff5-e4c354dccf1f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.492870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.493491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.493773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.494051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.494699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:13.495033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.496409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e77f0c-4209-4347-8d76-b7d0c3367d87 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.509960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 317/621] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:13.511117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd148f9f-9163-43fa-a634-c7f8a69925c5 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.511478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd148f9f-9163-43fa-a634-c7f8a69925c5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.511683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd148f9f-9163-43fa-a634-c7f8a69925c5 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.515915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 318/622] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:13.515915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd148f9f-9163-43fa-a634-c7f8a69925c5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.567139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fdcb6d-e529-4932-adce-502b73c99275 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.604217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.605079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.605568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.606057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.618441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.620226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-70bcaa12-67d0-444a-93a8-ed66d6c4a156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:13.636038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:13.637035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:13.637923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:13.637923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:13.637923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:13.640085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 305/623] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:13.641730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.642197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.642704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.653752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd148f9f-9163-43fa-a634-c7f8a69925c5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.665837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:13.667465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:13.673684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:13.694558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.695578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.696140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.696622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.704274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.706069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.718767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:13.719894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:13.720823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:13.721726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:13.724083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 319/624] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:13.725328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.763882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 306/625] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:13.765166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:13.901888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68890d5-a7f3-4971-b3ce-683e7a1b5f33 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.941201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:13.941813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.942094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.942365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.944021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:13.944588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:13.946185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfad8cd6-e43d-43b6-9a60-7c56e059b2ff None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:13.960033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 320/626] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:13.961677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cb65-8ef0-4a19-a92c-b1e38d03b24e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:13.962183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cb65-8ef0-4a19-a92c-b1e38d03b24e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:13.962629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43de4f1-4639-4f41-ae05-75ddcef2f5c7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:13.963250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928cb65-8ef0-4a19-a92c-b1e38d03b24e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:13.969366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 321/627] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:13.970384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928cb65-8ef0-4a19-a92c-b1e38d03b24e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.004347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.004954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.005236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.006029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.006815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:14.007122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.008601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24fb395-33d5-4ab7-9173-60642e471c9f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.015781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.028370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 307/628] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:14.028370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.028370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.028370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.044862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:14.046125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:14.076387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.094572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928cb65-8ef0-4a19-a92c-b1e38d03b24e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.124664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 308/629] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:14.125730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.147586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.148386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.148675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.148945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.158646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.160636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:14.175879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:14.177196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 322/630] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:14.179178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.246674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dd5271-de79-47bd-999c-a8a6f4818c72 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.283943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.284728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.285078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.285419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/61f9de1f0b754d74bf44b92eb2134081/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.287069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.293557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f5645-7bb9-458e-b70f-c71eda069c54 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.303682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=61f9de1f0b754d74bf44b92eb2134081, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.306304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.326663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 309/631] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:14 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/61f9de1f0b754d74bf44b92eb2134081/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:14.327967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.348761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.351888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.352644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.353025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.353419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.382867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:14.387894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:14.389041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:14.390007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:14.396453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 323/632] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:14.401512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.479095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be4f26-ffd4-4389-aca7-4093b2c1dbe2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.523806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.524443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.524724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.524995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.525721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:14.526027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.527452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.548420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 310/633] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:14.549239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.556409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea94a3b-5df1-41d7-8de7-4134ef2dba74 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.608373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.609016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.609550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.609831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.624604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:14.630744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:14.632688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:14.634162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:14.634810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.634884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 324/634] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:14.638347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.660950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.677856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e29c8-53dc-49c4-9add-c750628e5c6f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.725033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.725598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.725880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.726152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.726821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:14.727656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.729218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.740678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 311/635] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:14.742213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.753813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe447cfa-40fa-4eab-8746-d6870606580b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.791338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.791976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.792307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.792811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.859515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d17c8e-ee4c-4532-b92c-7fa524f61145 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:14.870945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.872788 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.894541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.896311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:14.896889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.897779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.898219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.898969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:14.899270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:14.900706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663cbc2-0496-4928-9113-a72f82b9b7dc None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:14.902340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:14.912329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 312/636] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:14.914775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807405-80f4-44da-9652-8563ff8779a8 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:14.915123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807405-80f4-44da-9652-8563ff8779a8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:14.915459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5807405-80f4-44da-9652-8563ff8779a8 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:14.921156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 313/637] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:14.921156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807405-80f4-44da-9652-8563ff8779a8 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:14.987847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 325/638] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:14.988574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.028330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5807405-80f4-44da-9652-8563ff8779a8 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.064660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.065288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.065562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.065834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.076598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.078037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.092180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83842e40-f1e5-4d64-92aa-90425dcc01d8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-028ca8e2-1746-4bbc-8ae5-c972e0277909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:15.092342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:15.093264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:15.094154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:15.094154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.094154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:15.094154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 314/639] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:15.094465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.094848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.094950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.112411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:15.113609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:15.114422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.135037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.135745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.136051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.136329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.146363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.147996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.158044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:15.164405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:15.165520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:15.166436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:15.166436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.166436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:15.166436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 326/640] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:15.166436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.191226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 315/641] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:15.192315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.281380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42f037-5350-4d3f-b6ea-40c2ef507083 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.319312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.320375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.320731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.321068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.330109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6e28d-5aff-46d1-9485-0cecf9bfd39a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.332058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.333573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:15.348117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:15.349364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:15.350500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:15.350500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:15.350500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:15.355842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 327/642] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:15.355842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.385207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.387230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.389224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.389224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.389224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.389788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:15.390097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.392025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.409029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 316/643] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:15.410628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.467561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.518323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ce772-36d2-408d-ae6d-a1fe06bc7203 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.560172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.560699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.560980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.561252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.562012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:15.562315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.563859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.566249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c7e6c-2623-41ab-b70a-eb8f3c07f5d1 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.582231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 328/644] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:15.583240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.621178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.622652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.623419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.623931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.624715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:15.625214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.626573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.640737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 317/645] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:15.641673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.693369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.718832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32af8d-d09b-41c5-8e30-900121bb5df1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.764798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.765436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.765714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.765983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.767846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:15.775562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3064496c-5234-4044-8924-252e7a18cf52 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.777719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.779207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:15.795358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:15.797006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 329/646] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:15.798183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.822183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.822882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.823171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.823440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.838291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:15.846234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:15.847315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:15.848395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:15.850697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 318/647] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:15.852337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:15.939523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6217be9-31fb-4e01-9a0e-9cfa2cf585f3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.971309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849a4d4-878d-4922-8fbc-50cd4764fa52 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:15.977738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:15.978377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:15.978659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:15.978927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:15.987975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:15.989425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:15.992706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:16.002771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:16.003738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:16.004736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:16.012138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 330/648] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:16.012138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.016690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.017336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.017741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.018018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.025676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.027132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:16.042073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:16.043565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:16.044732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:16.046487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 319/649] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:16.047495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.069274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.128949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e59b-ec43-4a39-b2fc-fed0d3462d73 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.154638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af404212-428b-48d0-9a0f-15b5de541f89 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.166540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.167339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.167838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.168372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.169379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.169829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.171527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.189037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.189634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.189910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.190180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.190833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.191133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.196693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.197203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 331/650] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.198346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.200807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 320/651] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.201593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.265883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.290213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.313430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f70b3c-a91a-4bb3-be7a-bd4a01220b54 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.313648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af927c66-6037-4bb8-b08e-8fdd16430519 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.350016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.350254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.350628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.350746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.350917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.351012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.351171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.351318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.352539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.352655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.352848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.353077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.354286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.354514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.368818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 321/652] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.369722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.373568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 332/653] 10.222.0.78 () {56 vars in 1137 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.373568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.472362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985c179-cfcf-42ae-a32c-f9c5ee81a1d5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.473816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699fc262-6c5b-4c3f-92b3-4c06c7d6f754 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.489178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.507188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.507620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.507917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.508815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.508979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.509165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.509283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.509447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.519679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.520467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.521874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-b4dc48df-fea0-4397-afbb-5329f0fc5fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:16.537570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:16.538657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 333/654] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564c7b-5b44-48a9-8e25-66ab2a2aa115 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564c7b-5b44-48a9-8e25-66ab2a2aa115 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11564c7b-5b44-48a9-8e25-66ab2a2aa115 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.545027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 334/655] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:16.546820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11564c7b-5b44-48a9-8e25-66ab2a2aa115 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.579968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.581447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.648608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11564c7b-5b44-48a9-8e25-66ab2a2aa115 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.677545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 322/656] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:16.678562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.682941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.683386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.683674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.683985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.684816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.685134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.686575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.692567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 335/657] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.693336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:16.729374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.730619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.798491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1bb4a-0e38-4959-b942-d83859461cb4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.817814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c61b6-2821-40d0-b79b-f44bdea1c19a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:16.840666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.841617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.843014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.843014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.857003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:16.857622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.857964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.858185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.858835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:16.859141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.860544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e976390-b20b-41d4-a05b-42df98705c29 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.873669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 336/658] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:16.874880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:16.875170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:16.875444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:16.894156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:16.895374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:16.926589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:16.928082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:16.971728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 337/659] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:16.972069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:16.973580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.000958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.071901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 323/660] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:17.071901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.089551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44964dca-fb33-4562-8bac-88277d9531b7 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.126213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.126842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.127122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.127395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.138077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.139610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:17.155641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:17.156679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:17.157569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:17.157569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:17.157569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:17.161485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 338/661] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:17.162335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.183117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3833cd6a-125c-412c-bb64-668c5414dd45 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.221495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.222737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.223227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.223702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.234811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.236653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.239659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.247963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:17.251977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:17.252901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:17.253840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 324/662] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:17.254477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.275956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8768454-af9a-4056-94d8-bba3c90ab28a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.312986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.313620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.313907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.314202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.326010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.327466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:17.342528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:17.343511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:17.344635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:17.344635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:17.344635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:17.347937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 339/663] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:17.349354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.369312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bbf3a6-4308-4c74-8cd2-7d916b5517e8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.406153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.407002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.407437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.407918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.408710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:17.409164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.410718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.424212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 325/664] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:17.425317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.456438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe1082c-f2d0-448b-920d-4d02ab2d90a0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.485922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.496880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.497525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.497807 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.498087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.501778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.509105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:17.514221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:17.515605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 340/665] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:17.520905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.554559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b613-fcc0-4f5c-9787-5231a215bd0a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.589264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130864-7018-4477-964d-736fe624f803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.589872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130864-7018-4477-964d-736fe624f803 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.590156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130864-7018-4477-964d-736fe624f803 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.590427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9130864-7018-4477-964d-736fe624f803 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.591149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130864-7018-4477-964d-736fe624f803 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:17.591454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130864-7018-4477-964d-736fe624f803 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.592925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9130864-7018-4477-964d-736fe624f803 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.612695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 326/666] 10.222.0.78 () {56 vars in 1139 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:17.613640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130864-7018-4477-964d-736fe624f803 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.630127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b51e0-6bcc-458c-b07b-baba7941790a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.669954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.670553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.670831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.671096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.671730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:17.672057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.673429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.692602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 341/667] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:17.692602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.723853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9130864-7018-4477-964d-736fe624f803 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.742809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.743340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.763651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.764734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.765016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.765448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.772795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.774062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:17.786062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:17.786957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 327/668] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:17.788318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.806283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82508228-7df9-42bf-8e5a-6cece2551029 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.845092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.846261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.846697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.847124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.848016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:17.848477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:17.850324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:17.860376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 342/669] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:17.861415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:17.898783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde2115-c48d-49d4-ac1a-03728b913417 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.933567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:17.934192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:17.934447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:17.934719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:17.970651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1706-b058-40f2-9a5c-19271913915e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:17.981136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:17.983213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.008222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.009065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.010316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.010500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.010600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.011007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.018204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.019589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:18.031153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:18.032085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:18.032971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 343/670] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:18.033479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.116956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 328/671] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:18.118966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.150090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc31d6d0-f1df-441c-a613-2f6ca7759623 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.188117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.188729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.189019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.189273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.189960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.190261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.191688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.204604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.206150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 344/672] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.207108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.233367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd815afa-c8cc-4cd5-bf7b-be1264a01ab3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.239845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.272143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.272703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.272985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.273258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.284131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.285589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:18.301096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:18.302120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:18.303649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:18.303649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:18.303649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:18.303649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 329/673] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:18.303649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.324204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266adfe5-7705-42c0-8082-7a1678ec4cb4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.361270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.361889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.362174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.362475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.363239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.363483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.365002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.377151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 345/674] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.377980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.420734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dbf8a4-7e0f-4ccd-b6fa-eb509b4397f5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.453829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.460215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.461118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.461201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.461437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.473590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.474721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.476062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.489125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789cebeb-a23f-4934-9cc6-34432654848c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:18.490733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:18.491631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:18.493116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:18.493116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:18.493116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:18.493116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 330/675] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:18.493116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.528397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.528935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.529219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.529492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.530218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=3061244943c84217bf9ba30224cd4951) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.530528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.531984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.547825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 346/676] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/role_assignments?user.id=3061244943c84217bf9ba30224cd4951&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.547825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.631082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902cf81c-4b7e-4d30-8183-a053b4959b56 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.679578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.680409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.680759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.681308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.683837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.683974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.685544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.685646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1e941-5399-41af-abaa-bbd1895c6cad None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.700832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 331/677] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.702753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.731967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.732597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.732889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.733166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.733903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=6b171220975d48488be864348cc9aca8) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.734218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.735880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364e055-395e-4ba1-8816-c21fa3939402 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.742090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 347/678] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/role_assignments?user.id=6b171220975d48488be864348cc9aca8&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.743114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3436-2178-4364-a5b9-061f8f9f65f9 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.743406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3436-2178-4364-a5b9-061f8f9f65f9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.743683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3436-2178-4364-a5b9-061f8f9f65f9 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.745695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 348/679] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:18.748676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed3436-2178-4364-a5b9-061f8f9f65f9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:18.754571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.770703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:18.846270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf7b77-0d20-48ef-bca4-6b921f3894ce None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.890535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.891152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.891429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.891698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.893144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:18.893338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.894723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b131b34-d674-4948-8f4e-55a5bd649131 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.900795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed3436-2178-4364-a5b9-061f8f9f65f9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:18.908288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 332/680] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:18.909535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.909826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.910091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.933399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:18.934594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:18.942255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:18.942940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:18.943225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:18.943494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:18.952854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:18.954523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:18.968266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:18.969249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 349/681] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:18.970184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.020047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 333/682] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:19.021045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.074294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.092057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.125110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a664f29f-d30a-43ad-85e2-6f014c8b9597 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.187131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabe5c4-9c38-4bc8-8ccc-bd210b875130 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.190543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.191186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.191470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.191742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.206236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.207710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:19.224154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:19.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:19.226000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:19.226985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 350/683] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:19.237987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.260660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c575-949a-441a-8592-77fb88063cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.261625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c575-949a-441a-8592-77fb88063cba None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.262047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c575-949a-441a-8592-77fb88063cba None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.262704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443c575-949a-441a-8592-77fb88063cba None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.264061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c575-949a-441a-8592-77fb88063cba None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:19.264539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c575-949a-441a-8592-77fb88063cba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.266181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443c575-949a-441a-8592-77fb88063cba None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.289505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 334/684] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:19.300732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c575-949a-441a-8592-77fb88063cba None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.364488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac959b87-1279-43bd-b845-f4862c62535f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.408208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.409241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.409622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.409998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.410939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:19.412114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.413561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.416662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.435844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 351/685] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:19.437667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.440243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.454280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443c575-949a-441a-8592-77fb88063cba None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.505171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.506083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.506539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.506998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.556522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84849c-d3fc-4048-9753-9c7b2a97f7c5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.583215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.585120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.595694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.596820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.596820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.597337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.609130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:19.613986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:19.615112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:19.616196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:19.623419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 352/686] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:19.624400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.688601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.691323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:19.705037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 335/687] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:19.705037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.745623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd4bad-8421-4d0e-b7f9-7711e55dca3e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.787581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.788632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.789137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.789619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.790652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=77e0d9362b3241d3b888e1cb0a2b5f4d) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:19.791162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.793926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.801440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 353/688] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/role_assignments?user.id=77e0d9362b3241d3b888e1cb0a2b5f4d&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:19.802629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.843452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348cf8a7-6b9d-44cf-8789-80639363b425 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:19.931721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:19.932736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:19.933262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:19.934486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:19.935460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:19.936060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:19.937728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:19.948463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 336/689] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:19.954747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:19.973339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b98fa-fd21-4242-99a5-f3cd73f654ed None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.004216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.030436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.031051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.031344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.031614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.034602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.043951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.045441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:20.061808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:20.062784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:20.063687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:20.063687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:20.063687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:20.063687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 354/690] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:20.068695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.093189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65daacf2-edd0-4f5b-8be0-63fa6110628d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.134467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.136121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.136634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.137668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.151668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.154051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:20.170319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:20.171323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:20.172244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:20.172244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:20.172244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:20.177372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 337/691] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:20.178273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.186685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ddacd-ca83-475d-ba8a-f4e8f6507672 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.229209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.229801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.230090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.230369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.231217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:20.231661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.234709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.260982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 355/692] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:20.268587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.307916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.320077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14934bd6-5b0a-4f9a-8e40-7501266ef777 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.324094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.371945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.373197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.373691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.374337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.375560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:20.376092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.377673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.392005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea13412-b054-4143-b3f4-1637e79d4df7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.396596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 338/693] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:20.405983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.449671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.450868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.451160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.451436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.466749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.468280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:20.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:20.495347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:20.496297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:20.502614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 356/694] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:20.507690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.601774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0191c3d-9ab8-43a2-b22b-527598bbc18f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.674951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.675598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.675932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.676215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.690004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.691465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.706528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:20.711681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:20.712927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:20.713921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 339/695] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:20.724265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.729493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963acf24-5f6b-4afc-8fe5-d7ea7f879bcc None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.758313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:20.771527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.772177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.772459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.772733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.773441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:20.773805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.775260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2612b-da99-4043-a24a-e85aaca57188 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.790421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 357/696] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:20.796777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306f4a-e09c-43c2-8801-b02ad1743848 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.797068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306f4a-e09c-43c2-8801-b02ad1743848 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.797346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e306f4a-e09c-43c2-8801-b02ad1743848 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.799004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 358/697] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:20.801016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e306f4a-e09c-43c2-8801-b02ad1743848 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:20.856606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bd8fd-d4b4-4aa1-a23a-ecafe2c23e2c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.895125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.895740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.896048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.896325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.897053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:20.897360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.898792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba6366-accb-4c8e-92c9-1ef3429aea83 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.916269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 340/698] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:20.917481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94638-7372-4f22-b9b2-15749e955c11 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.917769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94638-7372-4f22-b9b2-15749e955c11 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.918046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b94638-7372-4f22-b9b2-15749e955c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.937390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e306f4a-e09c-43c2-8801-b02ad1743848 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:20.937578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c6b94638-7372-4f22-b9b2-15749e955c11 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:20.938692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c6b94638-7372-4f22-b9b2-15749e955c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:20.979062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:20.979671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.980001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:20.980276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:20.981365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=61f9de1f0b754d74bf44b92eb2134081) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:20.981476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:20.983019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e11da9-2dd8-4194-ae29-ed08c695a3ea None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:20.989538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 359/699] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:20 2022] GET /identity/v3/role_assignments?user.id=61f9de1f0b754d74bf44b92eb2134081&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:20.998581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed04ea-ed6a-4957-9e50-eccd5c845a94 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:20.999746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed04ea-ed6a-4957-9e50-eccd5c845a94 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.000443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed04ea-ed6a-4957-9e50-eccd5c845a94 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.002888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 360/700] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:20 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:21.003991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed04ea-ed6a-4957-9e50-eccd5c845a94 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.023608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 341/701] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:21.023608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.023608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.023608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.050236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.050782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:21.053108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:21.069716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.175907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 342/702] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:21.187358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed04ea-ed6a-4957-9e50-eccd5c845a94 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.227676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.228368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.228702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.228983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.301109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.302496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.322107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.411213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 361/703] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:21.416980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.417547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.538705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789c4d9a-cb61-406d-b450-97ed763adae6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.554695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427c12f-2a69-497c-be3a-69c77b85332e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.576053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.585186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.585974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.586467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.586754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.598459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.599573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.600226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.600572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.600865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.602831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.605099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.612924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.614869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:21.622294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:21.624417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:21.625775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:21.625775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:21.625775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:21.625775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 343/704] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:21.626233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:21.641188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:21.643270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:21.644305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:21.644305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:21.644305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:21.644305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 362/705] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:21.644305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.742794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff2f6a5-29b1-44d6-9c30-aea1d8a2fc95 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.765101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a449c-768a-4d43-81bc-b0242614112e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.778960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.779659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.779965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.780236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.780876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.782255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:21.793726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:21.796573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:21.797829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 344/706] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:21.798787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.809352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.810599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.810989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.811330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.812543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:21.812915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.814758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.828755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 363/707] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:21.829741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:21.870993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.875611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:21.921735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad17ebf4-1a31-4c5e-8bd4-d026ab7134a3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.965483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:21.966638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:21.967052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:21.967513 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:21.972092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:21.972092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:21.972092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:21.978206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee899ccd-af6b-4403-8594-fe71a32d2f04 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:21.992824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 345/708] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:21.992824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:22.026124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.027141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.027141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.027601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.028375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:22.028762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.031030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.040568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 364/709] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:22.041523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:22.138173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ee086-b14d-4471-8903-78639ca2a815 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:22.143190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:22.153656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fd3e9-dadd-4f3c-a879-446e68ed727d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:22.163248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:22.189137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.190090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.190616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.191159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.193656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.194364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.194744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.195165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.195940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.197544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.199754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.201485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:22.216208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:22.217341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:22.218302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:22.224988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 346/711] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:22.226232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:22.227954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 365/711] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:47:22 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:22.228986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:22.367440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d10af8-82ac-4648-b835-9267e7f78663 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:22.377571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c20f6-ceeb-4042-9823-b1ba03177cf6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:22.406329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.407322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.407835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.408312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.409153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:22.409773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.411446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d4431-1389-4cfd-95e5-2cb31e6874a0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.417472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.418097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.418377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.418650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.419410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:22.419658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.420646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 347/712] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:22.422580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb2b4-4516-47d7-911a-0668cb93bb58 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.423096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb2b4-4516-47d7-911a-0668cb93bb58 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.423566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdcb2b4-4516-47d7-911a-0668cb93bb58 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.425589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 348/713] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:22.427261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.430780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.431418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.431985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.448794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 366/714] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:22.456328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:22.468886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:22.470609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:22.472469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:22.527528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:22.577351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 349/715] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:47:22.598657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e7f52-b091-45ad-b35f-e67d7c8ae01e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:22.635114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:22.635848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:22.636120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:22.636450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:22.637170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:22.637521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:22.639556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:22.646078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 367/716] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:22 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:22.901814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:23.028799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac3330-a85d-480f-9fe5-d49705b4fff1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:23.073920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:23.074815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:23.075245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:23.075671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] PATH_INFO: `/v3/services/network` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:23.076431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:23.078841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:23.093283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:23.094538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:23.095610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:23.098124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 350/717] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:22 2022] GET /identity/v3/services/network => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:23.106139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:23.201207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:23.237730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675eca3-0623-4ec1-b823-9571454f4849 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:23.279304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:23.280770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:23.281205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:23.281690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:23.282481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] RBAC: Adding query filter params (name=network) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:23.282954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:23.284538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed4301-929b-4633-9b7d-018639f28dfd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:23.289912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 368/718] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:23.295270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:23.337097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:23.391364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0045-50bd-471d-90ca-fe2fd0af0f78 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:23.391684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0045-50bd-471d-90ca-fe2fd0af0f78 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:23.391948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0045-50bd-471d-90ca-fe2fd0af0f78 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:23.393469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 369/719] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:23.401135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:23.401658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:23.402149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:23.423638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:23.425616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:23.461741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b9c2b-cf3c-41ea-abff-16219b1893b4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:23.516239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 370/720] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:23.553878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:23.554502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:23.554787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:23.555064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:23.555707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] RBAC: Adding query filter params (type=network) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:23.556041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:23.557418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:23.569642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 351/721] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:23.908192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:23.976548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:24.014997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcc54f-d64b-4af6-a16c-927880de2df1 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.052145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.053125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.053647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.054130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.065561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:24.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:24.071449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:24.072456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:24.079893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 371/722] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:24.081553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.182098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:24.185267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757f4ca-662d-4f45-9b49-c8c87728f3ff None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.221695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.222571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.222994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.223426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.224253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:24.224694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:24.226266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:24.239348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 352/723] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:24.240907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.348777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b35a10-82d8-4653-9ea5-01ba76cc2bca None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.380234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:24.388497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.389056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.389641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.389981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.401917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:24.403610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:24.419898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:24.420908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:24.422703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:24.422703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:24.422703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:24.426702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 372/724] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:24.428255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.532767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cdea3-3371-4553-9396-8b6d8c2ce098 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.567469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.568265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.568542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.568813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.569462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:24.569756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:24.571121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:24.588128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 353/725] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:24.591098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.699230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f39b4-54ea-477c-9a29-569e272c516a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.739996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.740603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.740886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.741162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.751607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:24.753099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-9a28eef2-7091-4daf-9900-38c52a543cf8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:24.764736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:24.765735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:24.767237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:24.770882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 373/726] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:24.772540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.800259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:24.885556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380e5b7-9ae6-4b90-b3b9-a02491fd99d8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:24.927996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:24.928982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.929267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.929541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.930771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:24.931289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:24.932841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:24.950382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43abe-8d3c-4009-be81-4ac607bf4199 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.950678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43abe-8d3c-4009-be81-4ac607bf4199 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.951099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb43abe-8d3c-4009-be81-4ac607bf4199 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.953292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 374/727] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:24.959571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23d5a-1463-4a11-9854-0e6605a7ea88 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:24.959890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23d5a-1463-4a11-9854-0e6605a7ea88 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:24.960147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23d5a-1463-4a11-9854-0e6605a7ea88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:24.963130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 354/728] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:24.972900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:24.983778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-26c23d5a-1463-4a11-9854-0e6605a7ea88 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:24.985623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-26c23d5a-1463-4a11-9854-0e6605a7ea88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:25.038158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.076437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.097660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 375/729] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 09:47:25.099559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ede2ce-7ffe-4b89-83f5-ffad980b18f3 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.100190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ede2ce-7ffe-4b89-83f5-ffad980b18f3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.100281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ede2ce-7ffe-4b89-83f5-ffad980b18f3 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.101656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 376/730] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:25.107880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce866ce-4958-4939-a3ac-4ed1dda4dfcc None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.107880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce866ce-4958-4939-a3ac-4ed1dda4dfcc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.107880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce866ce-4958-4939-a3ac-4ed1dda4dfcc None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.107880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 377/731] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:25.112402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa432f-2f28-4a55-957d-92aff40d15d6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.112679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa432f-2f28-4a55-957d-92aff40d15d6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.112944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa432f-2f28-4a55-957d-92aff40d15d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.119416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f52e-51ea-4729-b5f1-aebce1b84310 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:25.134845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f2fa432f-2f28-4a55-957d-92aff40d15d6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:25.136778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f2fa432f-2f28-4a55-957d-92aff40d15d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:25.167864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:25.169345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.169983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.170566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.231892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 378/732] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:25.231892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.231892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.231892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.269879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:25.273059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:25.321726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:25.323643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:25.410794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 379/733] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:25.420175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:25.496970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.521825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.567546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 355/734] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 09:47:25.572955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:25.607211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b183342-3703-4fcf-9631-af6ee5778627 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:25.648197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:25.648819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.649103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.649378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.650103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:25.651502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:25.677174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 380/735] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:25 2022] POST /identity/v3/services => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:25.678665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:25.724689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4d69-fb4d-4d4d-b75c-34e7ffd25a43 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:25.788635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:25.790438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.791024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.791309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.808223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:25.811266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:25.835377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:25.836408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:25.837304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:25.837304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:25.837304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:25.837304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 356/736] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:25.840544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:25.866368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.879541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:25.887640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947b292-e4d8-4f07-8523-d2c633b5070c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:25.942441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:25.942441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:25.942441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:25.942441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:25.957155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:25.965639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:25.966672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:25.967614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:25.971031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf06a58-7dc9-41de-9793-050eb74cd3ce None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:25.971937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 381/737] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:25.973328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.008494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.009294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.009643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.009988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.022422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:26.027932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:26.029852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 357/738] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:26.031943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.098135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225266ca-03d1-4c61-b5a2-3ca8832fb460 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.128651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.136886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.137505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.137792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.138941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.139606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.139941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.141702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56d33e-7337-4855-95f0-174bdc991e36 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.142626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.160580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 382/739] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.161484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.172775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.179305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.179967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.180599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.180599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.181231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.181542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.182981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.193715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 358/740] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.193715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.279014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093802cf-eee6-46ac-b4b8-2b416a61da57 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.309199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225e4ea-e96d-4f86-a781-6562cfd42545 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.316575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.317476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.317988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.318477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.319345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.319879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.321458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.334832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 383/741] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.336050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.349160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.349347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.349558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.349843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.357326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.358692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:26.371440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:26.372390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:26.373284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 359/742] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:26.373773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.394930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.427189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.467640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bace2a-81fd-4851-9e74-12cd3ce8ad7c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.501094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca341f-697f-4e04-ae48-6b5403cc5ba0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.504196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.504833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.505113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.505418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.516774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.519003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:26.534132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:26.535074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:26.536037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:26.536037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:26.536037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:26.536037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 384/743] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:26.536037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.543433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.544118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.544402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.544678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.556532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.558062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-49dbf042-aa15-4ed3-b6b0-5fd26a3dcacb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:26.573350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:26.574767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:26.576144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:26.576144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:26.576144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:26.579335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 360/744] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:26.581258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac30c9-cd84-43ef-9d4b-93168727ae47 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.581422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac30c9-cd84-43ef-9d4b-93168727ae47 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.581676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ac30c9-cd84-43ef-9d4b-93168727ae47 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.583099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 361/745] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:26.587659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac30c9-cd84-43ef-9d4b-93168727ae47 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.644785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2aa450-0efe-4744-9c94-c74e600de562 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.664550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.668998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.684448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.685159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.685444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.685716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.686410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.686715 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.688165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.701648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 385/746] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.702524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.716394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ac30c9-cd84-43ef-9d4b-93168727ae47 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.753918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.754529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.754812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.755089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.755741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.756074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.757462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30839951-aaed-45a3-a88f-503af91004c9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.769940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 362/747] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.771508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.771819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.772105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.794923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:26.796952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:26.816343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d611b9-852a-47af-91e7-5d507fe67bbf None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:26.855581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:26.856638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:26.856922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:26.857479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:26.858128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:26.858607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:26.860600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:26.870792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 363/748] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 09:47:26.871637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.884873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 386/749] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:26.885777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:26.937157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:26.949375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.011499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3431196-8c1f-4698-aaa9-9017f68288ec None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.018728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3265a3b5-7c2c-4d00-9271-cd4cc4f8d24b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.049051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.050395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.050818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.051228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.059426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.061080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.062383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.063347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.063726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.065270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.066188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=61f9de1f0b754d74bf44b92eb2134081) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:27.066602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.068165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.074446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 364/750] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/role_assignments?user.id=61f9de1f0b754d74bf44b92eb2134081&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:27.074627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:27.075641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:27.076627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:27.077155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.079364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 387/751] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:27.086848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.269861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237cd67-5cf1-4fa6-9822-68c96a89ec78 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.271335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.271496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35b824-b02b-4761-9ad3-f9e21023293f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.289695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.311460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.312468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.312818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.313363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.318133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.318826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.319177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.319522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.320366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:27.320750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.321815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.322471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.323253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:27.338056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:27.339029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:27.340025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 365/752] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:27.342198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.343202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 388/753] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:27.347475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.470880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3227d5af-ae0e-45e7-9725-660d50d20fab None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.476149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41bcb0-adfd-4eff-a293-6db341cdc04d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.507039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.507652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.507977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.508251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.508957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.510336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.514952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.515675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.515992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.516269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.517144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:27.517617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.518970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.536041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 366/754] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:27 2022] POST /identity/v3/services => generated 240 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:27.537367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.539364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 389/755] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:27.545236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.566853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.636125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.697362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e204-a788-4109-87f6-d670975967b7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.776165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.776786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.776996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c45649-3288-4d7d-ae90-c25ede949e6e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:27.777100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.777320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.823114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:27.823669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.824019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.824286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.836125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.837590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-35fc74a4-619e-4272-9172-ce8f7bca2088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:27.853145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:27.854369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:27.855279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:27.855279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:27.855279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:27.855279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 367/756] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:27.855783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffee78-1224-4655-b1e5-2967547fdbfd None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:27.856088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffee78-1224-4655-b1e5-2967547fdbfd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:27.856358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ffee78-1224-4655-b1e5-2967547fdbfd None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:27.857714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 368/757] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:27.862350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffee78-1224-4655-b1e5-2967547fdbfd None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:27.878673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:27.880772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b340b6-c13d-45e0-85dc-effb0d5d8df2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:27.900756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.970261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:27.994869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ffee78-1224-4655-b1e5-2967547fdbfd None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.005497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 390/758] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:28.007067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.007354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.007629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.033946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:28.036401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:28.037737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.038582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.039078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.039551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.040622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:28.041171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.042749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.061114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 369/759] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:28.062384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.124515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 391/760] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:28.125289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.202164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c3fe5b-ced9-4905-a88a-81782383a070 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.236176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.240913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.241816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.242316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.242792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.254145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.255879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.258704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:28.273749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:28.274731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:28.275617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:28.275617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:28.275617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:28.277783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 370/761] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:28.285098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.326061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bcd160-44c3-4024-bf30-18bbaf19e5fa None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.389892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.390548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.390829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.391103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.402612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.403045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec90aa-238c-4452-8156-eacc3ffd8f56 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.404096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:28.418561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:28.419511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 392/762] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:28.421111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.444752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.445710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.446193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.446685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.447633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:28.448186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.449772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.463840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 371/763] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:28.465304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.525791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.528938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bed69-8009-4195-9b56-df95e64e47e8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.540316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.572134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.572784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.573075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.573337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.574243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.575938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.577478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791afa2-9286-4f9f-8f63-aa31602846b1 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:28.588878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:28.589927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:28.590775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 393/764] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:28.591588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.621308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.622282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.622282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.622567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.623335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:28.623643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.625111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.637933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 372/765] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:28.638551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.713725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee4376-a85f-4a4e-a50f-7d641f0d99da None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.745751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd7f9c-45f2-4dfd-bb60-aab5bc8c6cb5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.752227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.753110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.753734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.754290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.762076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.763834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.764358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.785297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.785928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.786250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.786526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.787274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:28.787579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.789084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.795840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 373/766] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:28.796052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:28.905221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:28.906365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:28.907273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 394/767] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:28.908218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:28.912515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330a4179-7595-4954-b819-a8c6661cf44c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:28.942545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:28.950376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:28.951905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:28.951905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:28.951905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:28.953751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=7f645f811e804835a793f372a1ac7111, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:28.955244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:28.974329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 374/768] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:28 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:28.975321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:29.024767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:29.032141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3482bee4-f737-4340-9392-37fa773b51f3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.070836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.071890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.072384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.072888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.073732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:29.074337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:29.075969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:29.083092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 395/769] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:29.089183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:29.091470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29727d0-4725-43e7-8b83-d0a3ff467f4c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.131186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:29.135819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187af39-df4b-4bb8-b138-761d3567599d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:29.233299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:29.244530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e9bed5-f12a-48a7-a4ed-7033d60d8071 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.274272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 375/770] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:29.275748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaeaf3-0051-4af8-92af-e01882889a5a None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.276028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaeaf3-0051-4af8-92af-e01882889a5a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.276302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaeaf3-0051-4af8-92af-e01882889a5a None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.277987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 376/771] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:29.279597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaeaf3-0051-4af8-92af-e01882889a5a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:29.287964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.288804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.289234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.289679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.290542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] RBAC: Adding query filter params (interface=public, service_id=4c5ffdf5a6c44bf0932561b7c0c5119f, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:29.291001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:29.292590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910b580-efaa-4b67-9a56-289c3daa6446 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:29.301461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 396/772] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:29 2022] GET /identity/v3/endpoints?service_id=4c5ffdf5a6c44bf0932561b7c0c5119f&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:29.307312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.308049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.308550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.327381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:29.329403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:29.396997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:29.402562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaeaf3-0051-4af8-92af-e01882889a5a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.421976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 397/773] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:29.421976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:29.477038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.477666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.477951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.478226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.478959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b5793768f3fe46ab8b16908d5f570c91) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:29.479268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:29.486027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:29.498272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 377/774] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:29 2022] GET /identity/v3/role_assignments?user.id=b5793768f3fe46ab8b16908d5f570c91&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:29.552260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:29.569315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f727f33a-5c5d-4e96-b280-df6a6edda280 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.611592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.612263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.612546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.612820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.613691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:29.615395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:29.625625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 398/775] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:29 2022] GET /identity/v3/services => generated 772 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:29.716187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:29.830445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41b1d6-59c3-4bb8-80c1-3dfb94fd3579 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:29.868765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:29.869420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:29.869704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:29.869979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:29.883343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-2eef4ddb-82e9-49d8-b73c-82b58aa58830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:29.888234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:29.890345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:29.892359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:29.895826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 378/776] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:29.896148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.008635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5db47c-01b5-4269-89f1-3fd0f99eca59 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:30.052131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e107-fcd3-4a52-b900-d2d57ef4db16 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.052523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e107-fcd3-4a52-b900-d2d57ef4db16 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.052804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d86e107-fcd3-4a52-b900-d2d57ef4db16 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.054490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e4636-7755-446d-9148-a88436101e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:30.055114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e4636-7755-446d-9148-a88436101e1b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.055394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e4636-7755-446d-9148-a88436101e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.055668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e4636-7755-446d-9148-a88436101e1b None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.055888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 379/777] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:30.056381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e4636-7755-446d-9148-a88436101e1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:30.056685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e4636-7755-446d-9148-a88436101e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:30.058064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e4636-7755-446d-9148-a88436101e1b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:30.059855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.060148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.060420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.075970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 399/778] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:30.085039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:30.085263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e4636-7755-446d-9148-a88436101e1b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.086453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:30.106149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:30.115028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:30.167931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 380/779] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:30.256376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e4636-7755-446d-9148-a88436101e1b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:30.311096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:30.311726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.312036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.312311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.326675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:30.328162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:30.346181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:30.348872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:30.351018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:30.351018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:30.351018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:30.351018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 400/780] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:30.358321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.516866 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.546131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124057a4-2b73-4d9e-ad84-78a2bc7605b5 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:30.596983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:30.613590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:30.614244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.614522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.614815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.615430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:30.615724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:30.617126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:30.643157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 381/781] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:30.652866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.676449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:30.686587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba271810-b7c0-4fcf-8687-2ce0d1e7971d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:30.762751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa7e91-53d8-4e96-aac4-58e6624cc8ce None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:30.772068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:30.772983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.773458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.773877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.774620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:30.783551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:30.800421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:30.801445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:30.803777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:30.811116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 401/782] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/services/network => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:30.815230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:30.815230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:30.815230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:30.815230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:30.815548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.824938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:30.826393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:30.839029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:30.840327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:30.841601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 382/783] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:30.847902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:30.982161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff2479-bb21-48f5-a6c6-9997ed8789a0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.000701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.014037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.023438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.024300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.024739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.025433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.026623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] RBAC: Adding query filter params (name=network) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.027072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.028678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.034840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 402/784] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:31.042987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.061989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0091816a-bb40-4a75-90d7-ba9c4ca0198f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.148548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.149181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.149465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.149744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.150489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.150797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.152277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.162031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c895437-7010-487e-80dd-6007c0d9411f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.187453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 383/785] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:31.188650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.214784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.216363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.216887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.217355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.218732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] RBAC: Adding query filter params (type=network) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.219286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.221456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac0c4b-11cf-4fb8-8116-e7fa0700e402 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.230400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 403/786] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:31.231994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1c1a-5557-4712-b7bd-840c30ba9afa None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.232480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1c1a-5557-4712-b7bd-840c30ba9afa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.232944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b1c1a-5557-4712-b7bd-840c30ba9afa None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.234384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 404/787] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:31.239850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1c1a-5557-4712-b7bd-840c30ba9afa None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.313624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.320961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5173ab8e-68c7-41b0-92c3-fae72c3f7027 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.354824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.360835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.361903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.362387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.362927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.376692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.377598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b1c1a-5557-4712-b7bd-840c30ba9afa None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.379217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-d6039168-e2d2-49cd-91a2-eb7618d06b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:31.397344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:31.398336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:31.399280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:31.399280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:31.399280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:31.405953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 384/788] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:31.407743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.408257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.408786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.415312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.415947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.416225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.416493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.417486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] RBAC: Adding query filter params (interface=public, service_id=883e31364e3541efb80bcaebbcedadaf, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.417792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.420159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.430468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 405/789] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/endpoints?service_id=883e31364e3541efb80bcaebbcedadaf&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:31.430468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.435216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:31.436829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:31.533301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 385/790] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:31.534757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.547506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68580b82-cf8f-412a-b941-e5181476be8c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.571723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.587705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.588375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.588656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.588934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.589799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.590102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.591537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a0e3c-e5a8-4954-a12c-09bdbb33e643 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.606888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 406/791] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:31.611844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ffa4-f227-45ac-a00f-24bb9cd24713 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.611844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ffa4-f227-45ac-a00f-24bb9cd24713 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.611844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b9ffa4-f227-45ac-a00f-24bb9cd24713 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.611844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 407/792] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:31.616655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ffa4-f227-45ac-a00f-24bb9cd24713 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.666265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141eb5b-ebf5-444d-a802-fa7c40299323 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.681803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.704600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.705218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.705497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.705771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.706379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.707751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d2e89-7888-41e9-9e4d-db1d954d6a0c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.720120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 386/793] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:31.721589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.721880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.722157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.728852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b9ffa4-f227-45ac-a00f-24bb9cd24713 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.760012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:31.761313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:31.769386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.770087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.770372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.775474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.778992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.780613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:31.794659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:31.795620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:31.796580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:31.800843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 408/794] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:31.803095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.856500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:31.921271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 387/795] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:31.922166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:31.938597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c5272-acb6-4689-8b7f-2f73a6a17c1a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:31.980894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:31.981531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:31.981812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:31.982087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:31.982787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:31.983094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:31.984672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.003874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 409/796] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:32.006316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.057415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e2445-ad55-4c5b-aff1-00c0428fe308 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.075540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.128488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.129125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.129398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.129663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.150228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:32.160661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:32.161644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:32.163531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 388/797] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:32.176337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.178437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.187303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7d24f-5dad-42ec-95f9-d9e89bce3f2e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.244488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.245534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.246042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.246514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/61f9de1f0b754d74bf44b92eb2134081/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.248680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=61f9de1f0b754d74bf44b92eb2134081, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.250335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.265399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 410/798] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:32 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/61f9de1f0b754d74bf44b92eb2134081/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:32.266854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.313252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a16b-a4b2-4261-9b09-ffd6aa90e310 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.353217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.353850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.354133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.354409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.355104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:32.355407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.357128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e7c8e-40e6-4574-8855-46e3cb3f7378 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.366807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 389/799] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:32.368237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720e81-b6db-4585-96fa-c795a47382e9 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.368619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720e81-b6db-4585-96fa-c795a47382e9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.368894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e720e81-b6db-4585-96fa-c795a47382e9 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.370472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 390/800] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:32.372028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720e81-b6db-4585-96fa-c795a47382e9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.387121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0ca84-b6a8-4159-befc-e7a6ce5efca4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.415753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.429602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.430580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.431068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.431556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.442890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d2aa379c-7646-48f3-b974-5cedba8cc93d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:32.453105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:32.454113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:32.455018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:32.459273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 411/801] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:32.459273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.459273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.459273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.479982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:32.481184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:32.504134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.508666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e720e81-b6db-4585-96fa-c795a47382e9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.551716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.552384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.552688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.552968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.567817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.569307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 412/802] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:32.569428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.570260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:32.589493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:32.590484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:32.591388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:32.591388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:32.591388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:32.591388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 391/803] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:32.596187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.696251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.708472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d34339-1953-4120-99d3-5229d3dbe055 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.754586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.755387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.755779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.756815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.757575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:32.757985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.759459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.773065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 392/804] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:32.778131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.834107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ab7b6-ae50-4ad8-b46b-a0119cdb058c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.895932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33238820-e0ef-4034-a6a6-91b0f9eb7c24 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:32.907478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.908898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.909424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.910355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.911505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:32.912048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.914364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.926126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 413/805] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:32.927229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.947710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:32.948823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:32.949517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:32.949805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:32.950552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:32.961618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:32.963914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:32.979128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:32.980185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:32.981593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 393/806] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:32.982184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:32.985504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.070688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c8d19-1697-480e-8d27-7a7c30d57543 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.107998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fdc28e-324c-4d38-b524-2b754e653da5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.110480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.111289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.111727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.112213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.112949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.114535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:33.125146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:33.126412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 09:47:33.127412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:33.129846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 414/807] 10.222.0.78 () {56 vars in 1129 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:33.130875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.145908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.146580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.147451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.147726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.158915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.160399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:33.174951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:33.176125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:33.177112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:33.177112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:33.177112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:33.177112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 394/808] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:33.177336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.235281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.249737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.262267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddfdae-3b34-475a-9040-85e173675f3e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.298913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.299524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.299817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.300095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.300926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:33.301235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.302762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.310013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547dd5-1f47-49a7-aa2e-dad481982542 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.327251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 415/809] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:33.327251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.362465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.363316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.363740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.364178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.365226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:33.366569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.368113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.377790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 395/810] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:33.377790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.443183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9850510-5c1e-42a9-b28c-0070b1e970db None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.501206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.502677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.503234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.503908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.504747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:33.505386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.508253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.509677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbadd5-7786-4aef-a110-89530f17e47d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.510047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.525491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 416/811] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:33.526587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.532550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.556881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.558258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.558712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.559372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.643758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f181c-31df-435d-9be7-145ecc866e56 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.645560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.648545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.689337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.690226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.690899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.691327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.692962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:33.693433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.695593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.707217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 417/812] 10.222.0.78 () {56 vars in 1124 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:33.708764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.775390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 396/813] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:33.775390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.795123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.799894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:33.859921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699a46d-5dcd-47fe-bc6d-25879776c9d1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.889831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f3a37-dba8-4225-a5d4-b022d5c1c8cd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:33.905421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.906765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.907193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.907877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.918974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.921565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.929157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:33.930159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:33.930540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:33.930988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:33.931671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:33.933168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:33.936841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:33.937933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:33.939099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:33.946553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 418/814] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:33.946553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:33.974549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 397/815] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:33.974549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.069164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.082716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33998b3c-d679-46ef-9132-d415e525507d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.102936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.104894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3204aa-1393-41c1-b5d9-6707f98936c8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.124354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.125139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.125524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.125959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.139734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.141238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.145386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.145386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.145691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.145927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.146665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:34.146970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.148500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-c0736e66-d9b5-4c35-b012-a337aad138d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:34.160532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:34.161537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:34.162434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:34.162434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:34.162434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:34.162434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 419/816] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:34.163170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6a15-9338-4f22-9c56-58e7ed0fa9ac None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.163495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6a15-9338-4f22-9c56-58e7ed0fa9ac None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.163728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bd6a15-9338-4f22-9c56-58e7ed0fa9ac None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.168293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 398/817] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:34.169206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 420/818] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:34.174130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.174293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6a15-9338-4f22-9c56-58e7ed0fa9ac None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.289432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b32307-27b2-4354-815e-d4a24cc57776 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.301035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bd6a15-9338-4f22-9c56-58e7ed0fa9ac None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.317909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.327866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.328800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.329306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.329791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.330437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:34.330736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.332127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51f313-d8b6-4264-a565-b20ff8edfa04 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.340338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.341188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.341467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.341732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.350007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 399/819] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:34.350007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.350007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.350007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.354979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.356544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.367474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:34.368695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-a041fc43-8fc0-4274-9975-903f8b000d17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:34.371071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:34.372094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:34.373027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:34.373027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:34.373027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:34.376878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 421/820] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:34.377976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6a96-96f8-4c7e-af36-3a3a5c8de34f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.378262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6a96-96f8-4c7e-af36-3a3a5c8de34f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.378533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6a96-96f8-4c7e-af36-3a3a5c8de34f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.384616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 422/821] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:34.384616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6a96-96f8-4c7e-af36-3a3a5c8de34f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.410370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.446738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 400/822] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:34.450200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.521198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6a96-96f8-4c7e-af36-3a3a5c8de34f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.566811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.567454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.567736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.568038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.569564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67180bc-2c75-461f-94be-0d1908fa3363 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.575785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.577475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.588150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-26e91b30-309e-49f5-9f6f-0bf7e54a2bfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:34.591279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:34.594208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:34.595289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:34.597376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 423/823] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:34.598597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.598959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.599151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.609751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.610498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.610879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.611253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.612033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:34.612444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.613910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.620657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:34.622077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:34.626853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 401/824] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:34.632029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.704140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 424/825] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:34.705181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.728131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.739495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016e76b-492c-41c8-9d13-b8da3c5d38c2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.777922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.778973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.779698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.780227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.781362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:34.781821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.784169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.808136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 402/826] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:34.809779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.833045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88863d5f-2f97-4905-84b2-dc373347b104 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.863688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.871613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:34.872316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:34.872596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:34.872867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:34.880352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:34.881900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:34.894315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:34.895212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:34.896166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:34.900124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 425/827] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:34.901017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:34.951309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaada4b7-244c-42e8-b889-2f1d710b9504 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:34.982349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:34.998990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.000230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.000662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.001501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] PATH_INFO: `/v3/services/network` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.002516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.004551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.013554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428dd6c-9b08-42b4-9f11-1c01bdd224b7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:35.016397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:35.017395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:35.018249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 403/828] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:35.019068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.053545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.054149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.054510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.054925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.056135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:35.056450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.057871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.071820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 426/829] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:34 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:35.071820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.139938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd7ee-8831-4066-8cf8-e95449e6c372 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.143623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.190231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.191060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.191470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.191907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.192685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:35.193055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.194587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.208971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6937488-3b37-442b-81bb-6660c28881c8 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.215469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 404/830] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:35.216928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.250870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.251518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.251853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.252169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.255915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.267598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:35.276181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:35.277105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:47:35.277959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:35.279984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 427/831] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:35.280664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.354537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0fbfb-7f37-47dd-b478-b99e367a7f58 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.406391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.407520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.408356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.409081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.409919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] RBAC: Adding query filter params (name=network) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:35.410441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.410888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b558a60c-fbae-4e94-b684-8961e10645ea None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.412519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.419260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 405/832] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:35.420746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.448262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.448937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.449222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.449499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/b5793768f3fe46ab8b16908d5f570c91/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.459443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.466732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=b5793768f3fe46ab8b16908d5f570c91, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.468471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.491826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 428/833] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:35 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/b5793768f3fe46ab8b16908d5f570c91/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:35.491826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.592218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.599055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ed2ea-abd4-4137-9f01-02dbc20c658a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.633793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94c3c3-22e6-4300-9d19-10d625c184c5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.648939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.650074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.650554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.651051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.651985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:35.652492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.654102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.671539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 406/834] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:35.676759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.719607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.720646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.721309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.721793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.722667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] RBAC: Adding query filter params (type=network) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:35.722978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.725486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.734303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 429/835] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:35.742790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:35.843818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.877999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426aa6b-0fa0-4bb9-bd76-95adced84917 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.879996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c359cfe4-0af3-47ee-97bb-f97bb0790908 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:35.893317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:35.922887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.923410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.927840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.927840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.927840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.927840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:35.953329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:35.954336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:35.954870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:35.955265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 430/836] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:47:35.955588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:35.985137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:35.986709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-3a88902a-d7e0-45a0-962b-d46cf33ab5a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:36.008164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:36.009944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:36.010847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:36.010847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:36.010847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:36.014066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 407/837] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:36.015503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.093613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d640-1686-458f-94ec-82e2f75240e8 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.093831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d640-1686-458f-94ec-82e2f75240e8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.094091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6d640-1686-458f-94ec-82e2f75240e8 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.096065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 408/838] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:36.102445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.102730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.103266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.123604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:36.125041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:36.164779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c2e2f-afd4-47d8-849e-e7e5f31d7f73 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.210640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.211172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.211632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.211943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.212595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:36.212897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.214400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.222835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 409/839] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:36.229030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 431/840] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:36.241017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.325042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:36.363369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a727a47c-eeeb-44f0-8ae1-93d95cfc4f36 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.405935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.406579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.406863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.407140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.414902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.416407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:36.432602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:36.433640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 09:47:36.434608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:36.439571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 410/841] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:36.444909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.489951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:36.534345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.560910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23225235-4945-4d9a-8968-bd3751204597 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.596602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.597341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.597543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.597820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.598553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:36.598861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.600468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.619507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 432/842] 10.222.0.78 () {56 vars in 1135 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:36.626107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.667193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387fd62-6de1-4fc0-b483-e5515f9c7a3f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.705092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:36.712790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.713672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.714281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.714649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.715822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.717884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.719729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:36.730752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:36.732017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:36.733070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:36.739503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 411/843] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:36.740563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.760256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4a5db-2409-4208-aefd-0750de01382e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.798690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.799318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.799592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.799903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.871085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980b6f8-fe28-4f91-b41b-33b8415121e5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:36.876755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.878661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.909487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:36.910591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:36.911046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:36.911503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:36.912330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:36.912980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:36.914752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:36.925455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 412/844] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:36.926747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:36.968691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:36.992182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 433/845] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:36.998820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.010470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:37.057501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df76c6d-789d-4e31-a295-4a39522037a4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.093247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.093851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.094195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.094394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.095018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:37.095313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.096721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.108185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 413/846] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:37.137956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d6fab-c752-4c90-a453-4585245becf7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.178783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.179865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.180355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.180861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.193317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.195141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-52c5c331-939f-4361-8835-9072ca8217eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:37.212577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:37.213592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:37.214506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:37.214506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:37.214506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:37.221287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 434/847] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:37.223134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.332779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4decf-6aca-465c-b562-4b3ca3aa6fb8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.357902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:37.371574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.372537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.373039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.373583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.374481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:37.374981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.376636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.379950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b30d9-9d98-4064-be12-63615c255f0e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.380478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b30d9-9d98-4064-be12-63615c255f0e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.380961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b30d9-9d98-4064-be12-63615c255f0e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.385971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 435/848] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:37.388217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.388505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.388778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.392186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 414/849] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:37.398245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.413214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:37.414540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:37.446081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:37.511897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 436/850] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:37.526310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c83628-34eb-4cc8-8e8f-6a8a3d50a86e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.563322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.563982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.564260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.564531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.571988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.573435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-034c1d68-0951-4688-a8df-24a267bf5538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:37.584742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:37.585629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:37.586693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 415/851] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:37.594282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.706985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59356fa0-f2f0-417e-9849-6109c9df047d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.743238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.744738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.745035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.745312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.746081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:37.746394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.747312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58ea1-50a3-4748-b4e0-e7cd991051a9 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.747604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58ea1-50a3-4748-b4e0-e7cd991051a9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.747850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.748014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de58ea1-50a3-4748-b4e0-e7cd991051a9 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.749491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 416/852] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:37.754435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.754719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.754993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.761244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 437/853] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:37.768183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.775420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:37.776666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:37.797645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:37.819249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:37.854288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 417/854] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:37.855444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:37.894473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672dd7b-f83d-4762-bff8-7fd8021e8841 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:37.941977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:37.942678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:37.943023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:37.943382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:37.944407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=61f9de1f0b754d74bf44b92eb2134081) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:37.945229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:37.947150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:37.956718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 438/855] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:37 2022] GET /identity/v3/role_assignments?user.id=61f9de1f0b754d74bf44b92eb2134081&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:37.975331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e871d6-b89d-4f6f-81ab-f250c1e91540 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.015908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.017392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.018038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.018551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.019356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.021066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:38.032019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:38.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:38.033970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:38.036609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 418/856] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 24 09:47:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:38.045059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.136753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.144069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:38.162191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702f269-5eb6-4168-a068-2a35b9220c69 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.197901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:38.220295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.221258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.221961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.222386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.223619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:38.224364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.226694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.234327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 439/857] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:38.244395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.275234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f028e9-6733-4a17-9cb9-712b5af2c708 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.315155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.315700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.316116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.316477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.328932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:38.334132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:38.335202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:47:38.336082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:38.337045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 419/858] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:38.342352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.365021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1594c-bb01-43a0-957b-ecadb4149771 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.402922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.403900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.404393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.404844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.405755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:38.406236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.407861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.419952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 440/859] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:38.457990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e7c66-1d9b-4cce-8738-19eaf8af5723 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.458600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:38.500719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.501338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.501616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.501885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.502555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:38.502854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.504472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.515550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 420/860] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:38.521221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.541732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:38.637525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6984094-2b35-41ec-9ad4-2d88460d7c5f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.677447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.678088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.678364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.678632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.690816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.692352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:38.707475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:38.709251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:38.710491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:38.710491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:38.710491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:38.710491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 441/861] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:38.715618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.835401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86604fcf-9dff-4300-a7c9-3b59eb77c52a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:38.873095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:38.873758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:38.874037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:38.874307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:38.874947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:38.875243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:38.876635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:38.890599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 421/862] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:38.896123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:38.971603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.011431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f045db-a956-41ea-bf31-ebfb6f41d68e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.047663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.048422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.048770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.049113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.059031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.060606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-f41e4e95-d79c-49e0-86dd-bbd7579cd214 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:39.072509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:39.073812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:39.074986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:39.075891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 442/863] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:39.081089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:39.100858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.163871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b6657-4484-4ab3-8986-169032848518 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.164211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b6657-4484-4ab3-8986-169032848518 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.164917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b6657-4484-4ab3-8986-169032848518 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.167662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 443/864] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:39.174241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.174522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.174795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.194204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:39.195491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:39.199608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80872a8d-d1f4-4982-92cb-d00d59cb6648 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.242734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.243658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.244062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.245331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.246443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:39.246851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.248879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.270546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 422/866] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:39.271753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 444/866] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:39.276193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:39.355811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.380853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f94830d-4a72-4992-83cf-1da44a31960a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.417433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.418046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.418327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.418604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.491733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.493217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.549076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:39.583265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.608318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.620501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 423/867] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:39.626095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:39.657437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61be4435-b44e-4976-8ad4-131c71bbd9a9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.694916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.695744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.696149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.696567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.697447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.699545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.721130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 445/868] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:39 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:47:39.735547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bdb4c-c455-4c0c-a136-724782a028a0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.787215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.787909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.788195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.788474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.800110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.801651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:39.818580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:39.820582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:39.822743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:39.822743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:39.822743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:39.822743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 424/869] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:39.828908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:39.940503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9df06-4490-4575-85ca-949f0b169d2a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:39.957192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:39.977201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:39.978340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:39.978998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:39.980075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:39.981462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:39.981762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:39.983148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdb3449-411f-40a3-9c18-0c9e411daa7f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:39.996057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 446/870] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:40.001441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.017043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949ba-f68f-44d4-859e-8bc76bcad41e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.017360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949ba-f68f-44d4-859e-8bc76bcad41e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.017600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1949ba-f68f-44d4-859e-8bc76bcad41e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.019219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 447/871] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:40.024204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.024490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.024768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.033075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:40.044600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:40.045682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:40.122838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fce0f-372a-43f6-ae2f-8a19e10187c1 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.139941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 448/872] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:40.169673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.170698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.171184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.171697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.179216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.181199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:40.194126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:40.195018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:40.195938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:40.198855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 425/873] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:40.204806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.318015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525670-1e31-4bbb-90d3-24d65348722d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.326269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:40.355570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.356520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.356806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.357082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.357782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:40.358089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.359519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.375405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 449/874] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:40.381440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.430211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.491306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:40.516030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfd473-e657-4b6b-9b89-ca4b14c4f221 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.556493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.557103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.557379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.559153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.559907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b5793768f3fe46ab8b16908d5f570c91) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:40.560216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.561651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52252f-8258-4419-99eb-9e600b9da89b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.567675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 426/875] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/role_assignments?user.id=b5793768f3fe46ab8b16908d5f570c91&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:40.568843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646faae-1af6-41c3-a9f8-b55e469fee38 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.569121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646faae-1af6-41c3-a9f8-b55e469fee38 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.569390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646faae-1af6-41c3-a9f8-b55e469fee38 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.571821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 427/876] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:40.576631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.576913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.577404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.579414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8689315-121c-4bf8-ab03-c3ba5ef2bf5e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.596400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:40.597631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:40.626526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.628088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.628388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.628658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.629271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.630681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:40.643440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:40.644843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:40.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 450/877] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:40.653600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.692307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 428/878] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:40.692851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:40.765919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64059000-1749-41a2-a1e4-4092b2a23eb0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.801277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.801887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.802172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.802448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.803078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:40.803378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.804767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.811330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 451/879] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:40.817149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:40.908550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:40.932509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91609302-8d65-4087-b221-8d4227e26e33 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:40.969394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:40.970002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:40.970278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:40.970547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:40.971174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:40.971472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:40.972873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d48f77-4bf3-4bc7-9ec8-639ae080c550 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:40.977792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 429/880] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:41.015104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:41.122527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:41.129404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590f7d8-7e2d-4220-bbc9-1db05cb8d96e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:41.166704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:41.167995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:41.168302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:41.168579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:41.169237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:41.170686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3f2cd6-b647-46a7-9940-ad65e3e1d866 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:41.191933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 452/881] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:41 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:41.742804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320174-8562-4808-8922-fb5007cc8606 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:41.743326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320174-8562-4808-8922-fb5007cc8606 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:41.743326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1320174-8562-4808-8922-fb5007cc8606 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:41.744876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 430/882] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:41.750909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:41.751203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:41.751477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:41.770485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:41.771684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:41.853415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 453/883] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:42.127526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1320174-8562-4808-8922-fb5007cc8606 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:42.233643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1320174-8562-4808-8922-fb5007cc8606 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:42.268948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:42.269585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.269866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.270139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.270716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:42.272120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:42.281592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:42.282584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:42.283439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 431/884] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:42.288900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:42.374117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:42.395504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027983-fe91-4717-b9eb-8cbca64156e0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:42.436784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:42.437396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.437678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.437951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.438585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:42.438889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:42.440302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:42.445404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 454/885] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:42.453310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:42.468754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:42.568249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd4093-a0ea-4cdd-b5f4-9ea6f0bb8dc3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:42.613243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:42.613856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.614135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.614408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.615041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:42.615341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:42.616764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242a0e08-8b82-4880-b4ec-589541b3d29f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:42.623478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 432/886] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:42.633277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:42.740276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fc073-3a6b-4853-9a75-21a6efa78dae None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:42.769595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a27ed-69af-4084-9104-bdf9d339ddef None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.770112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a27ed-69af-4084-9104-bdf9d339ddef None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.770191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a27ed-69af-4084-9104-bdf9d339ddef None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.771626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 433/887] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:42.779134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288b8b-933b-4309-8b77-05fdcebc0f2b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.779423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288b8b-933b-4309-8b77-05fdcebc0f2b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.779693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22288b8b-933b-4309-8b77-05fdcebc0f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.786566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:42.787825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.788255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.788766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.789624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] RBAC: Adding query filter params (interface=public, service_id=74cd85b17e554d83a5a70dbe3656bb40, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:42.790081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:42.791595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:42.802032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-22288b8b-933b-4309-8b77-05fdcebc0f2b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:42.802032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-22288b8b-933b-4309-8b77-05fdcebc0f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:42.802559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 455/888] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:42 2022] GET /identity/v3/endpoints?service_id=74cd85b17e554d83a5a70dbe3656bb40&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:42.803838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:42.817027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:42.888313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:42.899816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 434/889] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:42.927573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db6c7b-51b3-4048-a692-d22dbd9eb4b9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:42.964401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:42.965538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.965946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.966219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.966837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:42.969146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e429cd-1e8e-41fe-8091-badfc01b3268 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.969443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e429cd-1e8e-41fe-8091-badfc01b3268 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.969719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e429cd-1e8e-41fe-8091-badfc01b3268 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.970612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:42.971320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 435/890] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:42.976850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f6e3-a658-4484-a33f-79a191f7770b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:42.977442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f6e3-a658-4484-a33f-79a191f7770b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:42.977758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6544f6e3-a658-4484-a33f-79a191f7770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:42.977910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 456/891] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:42 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:47:42.999245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6544f6e3-a658-4484-a33f-79a191f7770b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:43.000650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6544f6e3-a658-4484-a33f-79a191f7770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:43.076688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 436/892] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:43.178198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.251781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb367ca-7abb-4979-abc9-415eeaf91b7c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.252115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb367ca-7abb-4979-abc9-415eeaf91b7c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.252298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb367ca-7abb-4979-abc9-415eeaf91b7c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.254379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:43.255608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 437/893] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:43.259067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.259355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.259681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.284106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:43.285600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:43.290342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf1158-4a61-4281-bad4-10a335980dc2 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.332189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.332826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.333109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.333384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.344981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:43.350065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:43.351000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:43.351882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 457/894] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:43.357650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.363553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 438/895] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:43.365898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.429632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:43.474718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733d5f-1038-4d78-92fd-893cb853a6a4 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.488308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a316360-a570-4fc5-8c75-d52e24d7611a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.521202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:43.521655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.522302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.522569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.523137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.523983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:43.524407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:43.527247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:43.529579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.530445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.530919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.531430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.532310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:43.533871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:43.538815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 439/896] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:43.540866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.557143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 458/897] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:43 2022] POST /identity/v3/services => generated 241 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:47:43.635701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.682545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b080e8-3863-40ff-9fa7-317d6d7f1a08 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.712181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:43.732836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.733470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.733753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.734035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.745456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:43.748013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:43.763549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28676e-f0ce-41cf-b73f-3e21a4946cc6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:43.764235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:43.765361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:43.766570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:43.766570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:43.766570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:43.766570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 440/898] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:43.771355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.812372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.814261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.814825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.815381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.816191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:43.818914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:43.830964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:43.832173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:43.833268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:43.841218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 459/899] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:43.842788 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.899841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfd760-18d3-495f-9ad2-70ad71d7cc9d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:43.922823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:43.943569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:43.944206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:43.944481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:43.944749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:43.945391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:43.945689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:43.947134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:43.967165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 441/900] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:43.968224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:43.980657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99f8d1-b298-4253-a2b4-cdc4263d6786 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.002404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.026264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.027133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.027561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.028011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.030052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:44.030517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.032536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.044453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 460/901] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:44.048642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.095522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830aea77-7b56-4f18-91a6-0074641979a5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.142099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.143221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.143893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.144387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.154650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.156911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.161179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba44761b-7438-42ea-b005-9368b0569dd5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:44.175086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:44.176269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:44.177442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:44.179904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 442/902] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:44.189400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.191981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.214511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.215434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.215960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.216477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.217422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:44.217980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.219695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.225609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 461/903] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:44.232178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.288998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.304784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128527b9-41fe-4a3f-ab63-9e15982ad95a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.341918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.342341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.342621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.342894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.343672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:44.344009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.345497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.368814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 443/904] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:44.369893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.376506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b0b5e-7b41-4913-8019-4f2c85829069 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.425428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.426392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.426897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.427888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.429083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] RBAC: Adding query filter params (interface=public, service_id=2cdcb6171ab24c8a8f32d9029a616996, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:44.429590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.432027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.441051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 462/905] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/endpoints?service_id=2cdcb6171ab24c8a8f32d9029a616996&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:44.447478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.489265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00476f4f-72ee-4217-aff0-f93c4a340f02 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.512847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.527238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.527884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.528159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.529139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.549929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.573928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55763179-9e28-4c62-9f69-b1637fcd6fac None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.608523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.610054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.612416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.613265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.613752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.614227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.615045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.616645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.624920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 463/906] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:47:44.737706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 444/907] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:44.744126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.857766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca607df4-5bab-42bc-b7e4-2d7ebead257b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:44.872365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:44.896883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:44.897852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.898330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.898805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.910605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:44.912356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-19e860a6-8edc-4e0b-8458-8feb8c34a6f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:44.928669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:44.929704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:44.930663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:44.930663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:44.930663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:44.932748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 464/908] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:44.940008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:44.981111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b38b6-ec55-4e73-b179-fe01a6b2fc60 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.981779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b38b6-ec55-4e73-b179-fe01a6b2fc60 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.982542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b38b6-ec55-4e73-b179-fe01a6b2fc60 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:44.984449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 465/909] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:44.994291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:44.994588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:44.994863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.001266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.017592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:45.018823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:45.067139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab5db79-b43d-45ee-bdd4-220c84d982ef None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.103817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 466/910] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:45.117630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.118284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.118576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.118904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.119878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:45.120188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.121631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.134968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 445/911] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:45.140622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.242859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.252312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dec55f-e156-40f0-af1b-59d1e9f6ec55 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.290605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.291617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.292139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.292630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.300311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.301985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:45.313778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:45.314677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:45.315659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:45.318013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 467/912] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:45.326270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.422300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.439949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075dd19-5801-4cea-a931-f898d3eadf2b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.453273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.493160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.493780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.494063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.494592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.495296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:45.495608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.498403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.515952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 446/913] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:45.521305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.549258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbf9081-8ef0-4645-adb0-cf50dfcfd789 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.586645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.587287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.587569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.587868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.588484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.589889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-2144729f-09ef-4d72-aa0f-9c7087216911 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:45.598867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:45.599910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 09:47:45.600770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:45.608011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 468/914] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:45.608011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8f2d-f2ec-44f2-a680-6f5ce2ec1ea6 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.608011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8f2d-f2ec-44f2-a680-6f5ce2ec1ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.608011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d8f2d-f2ec-44f2-a680-6f5ce2ec1ea6 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.608011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 469/915] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:45.609113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d8f2d-f2ec-44f2-a680-6f5ce2ec1ea6 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.629324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.632859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465b5e83-a8d9-43c2-b672-5f9e253857b6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.669401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.670545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.670823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.671094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.671835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] RBAC: Adding query filter params (role.id=a3a0c05005d2415ab6cfe90f7a7f7792, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b5793768f3fe46ab8b16908d5f570c91) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:45.672156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.673643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5fd3f-f27f-4315-b8ba-a9310316eecd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.679583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 447/916] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/role_assignments?user.id=b5793768f3fe46ab8b16908d5f570c91&scope.project.id=35e2575a816145b297430e41f586390d&role.id=a3a0c05005d2415ab6cfe90f7a7f7792 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:45.680865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.681140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.681437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.704659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:45.706415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:45.734434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d8f2d-f2ec-44f2-a680-6f5ce2ec1ea6 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.745636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.784139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.785278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.785814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.786297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.787135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:45.787929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.789737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.796237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 470/917] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:45.801651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:45.805005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 448/918] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:45.908597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:45.928830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640aeed8-cb67-4adf-b996-09fd1a980701 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:45.965814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:45.966426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:45.966753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:45.967155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:45.967800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:45.968114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:45.969482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:45.974864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 471/919] 10.222.0.78 () {56 vars in 1114 bytes} [Wed Aug 24 09:47:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:45.980383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.087268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3a663-9b8f-45f2-804d-20d1b5984663 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.091553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.125772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.126391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.126671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.126945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.127608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.129015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.153712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 449/920] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:46.211248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7da6e-242e-4f20-87c1-43e3ceba453a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.263440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.264314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.264594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.264868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.265464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.266988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:46.277027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:46.278146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:46.279745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 472/921] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:46.287465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.308695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:46.399183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9a5da-5da9-481d-b9f5-717dd610a3f9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.434759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.436377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.437274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.437775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.438604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:46.439128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.440833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.461590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 450/922] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:46.463816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.474894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:46.570024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb8d0e-5589-4f43-b7cf-deba98830184 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.604969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.605577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.605856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.606129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.606761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:46.607063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.609189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277637d-7b19-44ac-a7b6-f9f4073d2817 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.619213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 473/923] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:46.620574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.710261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:46.725628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426acdc-ba1b-47f7-a7d9-b23b105c5263 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.725931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426acdc-ba1b-47f7-a7d9-b23b105c5263 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.726322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426acdc-ba1b-47f7-a7d9-b23b105c5263 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.727700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 474/924] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:46.732313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86243ca-2021-4918-b6a3-a94826113fe4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.732789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.733079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.733425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.753071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:46.754385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:46.774777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.775398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.775679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.775986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.776696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] RBAC: Adding query filter params (interface=public, service_id=29e1215ce1f4429ba0c4106dbb1c4042, region_id=RegionOne) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:46.777003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.778377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.786379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 451/925] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:46 2022] GET /identity/v3/endpoints?service_id=29e1215ce1f4429ba0c4106dbb1c4042&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:46.792190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:46.830220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:46.839886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 475/926] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:46.910404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a59d7a-3d94-4f32-bd6a-559b2d7cdf9f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:46.945757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:46.946369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:46.946649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:46.946925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:46.947688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:46.949146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:46.956201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 452/927] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:46 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:47:47.131884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:47.237887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f827bf-20c3-40b0-b35b-55352dc01852 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:47.278559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:47.279586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.280860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.281426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.282234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:47.283869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:47.293314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:47.294299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 09:47:47.295208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:47.298576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 476/928] 10.222.0.78 () {56 vars in 1132 bytes} [Wed Aug 24 09:47:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:47.304166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:47.319004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:47.412910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ef3c29-7f93-48cc-a0b7-8fa2f105896b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:47.451001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:47.451714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.452804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.453228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.453872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:47.454180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:47.455565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:47.460927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 453/929] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:47.466580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:47.533321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:47.574631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc456cf-4649-498e-9bb6-b03e065767a5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:47.614974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:47.616041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.616545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.617028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.617863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:47.618370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:47.619985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc181fc-7180-4752-b42c-608ee70ca81f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:47.626523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 477/930] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:47.632003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:47.661572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f33d18-6780-4eed-a228-4e0638a05b9d None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.661871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f33d18-6780-4eed-a228-4e0638a05b9d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.662126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f33d18-6780-4eed-a228-4e0638a05b9d None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.663763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 478/931] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:47.668382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.668665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.668935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.689552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:47.691160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:47.716209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:47.742383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859c3b2-929c-4db5-8874-eae5ba81b959 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:47.780686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:47.781302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:47.781587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:47.781863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:47.782538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:47.784000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:47.784123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 479/932] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:47.814927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 454/933] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:48.056593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.119000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:48.162326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61796285-9af4-4a12-bc0d-f3f1563326b8 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.204055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.204689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.204965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.205233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.215438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-8dbfd8eb-56b6-47c0-97a2-41579ae057b2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:48.220261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:48.222385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:48.224437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 480/934] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:48.229109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.332160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3859976-cb66-4766-af12-1f4d5fbaf91a None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.332785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3859976-cb66-4766-af12-1f4d5fbaf91a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.333297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3859976-cb66-4766-af12-1f4d5fbaf91a None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.335125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 481/935] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:48.338950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69ed9b-1e54-42c0-a505-c92eeb66b247 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.340736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.341233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.341718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.365517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:48.367573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:48.368193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:48.376432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.377046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.377324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.377594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.378755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:48.379134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:48.380542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:48.394509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 455/936] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:48.400463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.468147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 482/937] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:48.526668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736981ee-9cae-4a9f-bba7-dcb053352186 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.564758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.565445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.565720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.565994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.577988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:48.579829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:48.596177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:48.597190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:48.598876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:48.598876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:48.598876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.598876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 456/938] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:48.603090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.652829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:48.711585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f52c333-aff3-48ef-93ef-dc18fe8ded8f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.747389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.747578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.748177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.748487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.748735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.750122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:48.750635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:48.752090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:48.769561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 483/939] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:48.774976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.803669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:48.868708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09c84ae-dcfe-4f43-8642-b8255afeb12b None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.884524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e848d-d1d9-44c5-a7b7-6d7f78700b0c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:48.887220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:48.910523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.911601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.912117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.912615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.913559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:48.915245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:48.918312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING keystone.server.flask.common [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:47:48.922547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:48.923208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.923488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.923845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.932366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:48.933793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:48.943341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.sql.core [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:47:48.943341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:47:48.943341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [parameters: {'id': 'ae2eb97cc1d24f8ab691644a17417d54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:47:48.943341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81467) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:48.948351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:48.949330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:48.950315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:48.955146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 484/940] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:48.956190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0a7630bb-3f1f-46f4-b7ed-ab2aae71fc7e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae2eb97cc1d24f8ab691644a17417d54.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae2eb97cc1d24f8ab691644a17417d54. Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:47:48.967212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application result.read() Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 09:47:48.968144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application yield resource Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise self.value Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application session.commit() Aug 24 09:47:48.969288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:47:48.970233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:47:48.971148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application result.read() Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:47:48.973369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [parameters: {'id': 'ae2eb97cc1d24f8ab691644a17417d54', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.975490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:48.977477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ae2eb97cc1d24f8ab691644a17417d54. Aug 24 09:47:48.979652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:48.987920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 457/941] 10.222.0.78 () {60 vars in 1140 bytes} [Wed Aug 24 09:47:48 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:48.989746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d01a-8605-45db-b031-31861b5edf75 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:48.990032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d01a-8605-45db-b031-31861b5edf75 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:48.990303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d01a-8605-45db-b031-31861b5edf75 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:48.995075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 458/942] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:48.996246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849d01a-8605-45db-b031-31861b5edf75 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.064877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dfd04-5cdc-42ad-9fa6-d3399da0b267 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.103565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.105297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.106013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.106202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.106529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.107211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:49.107559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.107689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849d01a-8605-45db-b031-31861b5edf75 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.109602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070d6ded-c9bf-4687-b802-cac76650afbe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.129470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 485/943] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:49.130939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.131226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.131497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.148283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.148921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.149203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.149479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.149714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:49.150894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:49.156925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.158504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.168430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:49.170183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:49.171101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 459/944] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:49.173059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.227775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 486/945] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:49.232709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.285367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c684edea-7fc1-4700-8910-fd58a09bbda6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.323344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.324008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.324289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.324563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.336848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.338362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:49.353882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:49.355292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:49.356225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:49.356225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:49.356225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:49.360036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 460/946] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:49.360950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.373434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f6bed-4fc5-4a4c-8831-03ce32653346 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.377940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.423354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.424954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.425388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.425830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.426895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:49.427752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.429909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.456869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 487/947] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:49.478415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd1656-0808-4432-8f24-57bff6371540 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.498485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.516489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.529066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.529706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.529987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.530261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.531503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:49.531831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.534258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.553270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 461/948] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:49.558371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.635072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e2568-014a-409d-a08e-4b84f58cfc54 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.655652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.681322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.681890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.682218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.682499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.683501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.684524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.693755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ad906-973c-4743-af1a-f34747746b71 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:49.695220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:49.696246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:49.697129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 488/949] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:49.702306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.738285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.738928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.739203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.739472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.747148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.748628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:49.761187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:49.762166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:49.763058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 462/950] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:49.768333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.802785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.829654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcfd9e-d7bc-4b0f-bffc-90b29f6c04c1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.868605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.869217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.869492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.869769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.870408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:49.870718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.872127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56cb7c9-5a9b-48c1-825b-f7fb69628a0e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.877828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 489/951] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:49.879044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80100a0d-275f-4444-8dd2-852e30fb3466 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.879328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80100a0d-275f-4444-8dd2-852e30fb3466 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.879604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80100a0d-275f-4444-8dd2-852e30fb3466 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.881130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 490/952] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:49.883234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80100a0d-275f-4444-8dd2-852e30fb3466 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:49.894351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f152c-1cd1-45b4-9830-3c531a628b28 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:49.925072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:49.931016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567377f-835d-4577-935a-0063d633325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:49.931684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567377f-835d-4577-935a-0063d633325a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.931962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567377f-835d-4577-935a-0063d633325a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.932236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567377f-835d-4577-935a-0063d633325a None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.932930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567377f-835d-4577-935a-0063d633325a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:49.933236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567377f-835d-4577-935a-0063d633325a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:49.934675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567377f-835d-4577-935a-0063d633325a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:49.953609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 463/953] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:49.953609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:49.953609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:49.953609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:49.970716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:49.972276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:50.001711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80100a0d-275f-4444-8dd2-852e30fb3466 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.038339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.038953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.039235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.039509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.040179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:50.040589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.041960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.055816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 464/954] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:50.055816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:50.056064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 491/955] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:50.056064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:50.059329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:50.166541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364bf89-c228-4390-9831-53feb2340f9e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.168260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c89279-63a3-4071-96a1-4646a40813fe None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.190645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:50.204416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.205158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.205529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.205897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/b5793768f3fe46ab8b16908d5f570c91/roles/a3a0c05005d2415ab6cfe90f7a7f7792` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.206881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.207553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.207923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.208127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=b5793768f3fe46ab8b16908d5f570c91, role_id=a3a0c05005d2415ab6cfe90f7a7f7792)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.208312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.209034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.209635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.210495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.225798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 465/956] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:47:50 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/b5793768f3fe46ab8b16908d5f570c91/roles/a3a0c05005d2415ab6cfe90f7a7f7792 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:47:50.240052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 492/957] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:50 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:50.337345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:50.460912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b41101d-b6f1-4a8f-b887-c9fa288e4071 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.507061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.508000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.508432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.508879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.509606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.511113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:50.520493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:50.521850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:50.522952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:50.525501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 466/958] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:50.531899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:50.602701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:50.651427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787951e-1c75-4aaa-a9ef-837037c18af3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.689443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.690116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.690402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.690674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.691310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:50.691614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.693028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d719b-14b6-4b99-8d00-6d343c974e67 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.704016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 493/959] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:50.705463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:50.752248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:50.812462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f56176-2a8b-4133-b4b1-ff2fc52e8e57 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:50.850521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:50.851316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:50.851855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:50.852364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:50.853151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:50.853460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:50.854838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48be78db-2a50-4838-8279-44ee4158cc5e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:50.865566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 467/960] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:51.562388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6dff-2a66-467b-8c9d-8a2d3c8202d9 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:51.563533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6dff-2a66-467b-8c9d-8a2d3c8202d9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:51.563817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc6dff-2a66-467b-8c9d-8a2d3c8202d9 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:51.566211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 494/961] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:51.571983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:51.572494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:51.572986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:51.596772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:51.597979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:51.665294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 468/962] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:51.935619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc6dff-2a66-467b-8c9d-8a2d3c8202d9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.048585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc6dff-2a66-467b-8c9d-8a2d3c8202d9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.073068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.084667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.085278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.085556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.085831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.086479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:52.088019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:52.090560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING keystone.server.flask.common [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 09:47:52.105257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.sql.core [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:47:52.105257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:47:52.105257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [parameters: {'user_id': '91d3b699a7054ac482331e8657ff2e00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:47:52.105257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81466) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-5bd23139-4843-47e1-ace2-7042d2c757cd None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result.read() Aug 24 09:47:52.113352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:47:52.114213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application yield resource Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application next(self.gen) Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise self.value Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application session.commit() Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 09:47:52.115582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.session.flush() Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application compat.raise_( Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 09:47:52.116891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 09:47:52.117813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application conn.query(q) Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application result.read() Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 09:47:52.118679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [parameters: {'user_id': '91d3b699a7054ac482331e8657ff2e00', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:52.120339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 09:47:52.122259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.124560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 495/963] 10.222.0.78 () {60 vars in 1131 bytes} [Wed Aug 24 09:47:51 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:52.130940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.162235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.229396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114e84-7066-40fc-8084-799f5af0a400 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.229701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114e84-7066-40fc-8084-799f5af0a400 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.229872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86114e84-7066-40fc-8084-799f5af0a400 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.230983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3bd1f-bb24-40aa-ae29-51e386931e2f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.234634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 496/964] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:52.241606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4da3-ac3b-4253-998e-a453eeb8c595 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.242030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4da3-ac3b-4253-998e-a453eeb8c595 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.242457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d4da3-ac3b-4253-998e-a453eeb8c595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.261533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-178d4da3-ac3b-4253-998e-a453eeb8c595 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:52.263440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-178d4da3-ac3b-4253-998e-a453eeb8c595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:52.268316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.268947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.269223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.269490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.277368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:52.278921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:52.291882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:52.293512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:52.294929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 469/965] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:52.301168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.359883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 497/966] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:52.396408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066b22-f0f9-4e6e-9e68-5678d48d5baf None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.396705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066b22-f0f9-4e6e-9e68-5678d48d5baf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.396990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c066b22-f0f9-4e6e-9e68-5678d48d5baf None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.398608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 498/967] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:47:52.403579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.403900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.404183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.422270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:52.424253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:52.427516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.435709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e214e99-9dad-453b-acd6-9269f3a8d715 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.475978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.476967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.477466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.477963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.479086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:52.479651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:52.481320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:52.499733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 499/968] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:52.519694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 470/969] 10.222.0.78 () {56 vars in 1101 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:52.671241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.723008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.777764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.784522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d3e480-4a34-49d3-9fb8-0603cc409c6b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.822522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.823968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.824248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.824519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.836825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:47:52.842651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:52.843586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:47:52.845303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:52.846488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 500/970] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:52.850578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.893934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d198463-380a-40fd-8188-1ac9b472a6e7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.927284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.927951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.928229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.928497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.929077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:52.930434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-b8b164c2-c265-4c98-9ab3-6302dd9a2fa9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:52.940724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:52.941928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:52.942769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 471/971] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:52.943561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87df-cc73-4c2e-b6e9-7e804be214fd None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.943561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87df-cc73-4c2e-b6e9-7e804be214fd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:52.943693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c87df-cc73-4c2e-b6e9-7e804be214fd None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:52.944886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 472/972] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:52.947556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c87df-cc73-4c2e-b6e9-7e804be214fd None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:52.962195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d81f-82fc-4413-96a6-b99c6cbe52c0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:52.966989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.969965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:52.999041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:52.999647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:52.999967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.000240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.001445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.001741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.003239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750a0946-3ae1-4eae-96c1-7c8409f6f53f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.011539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 501/973] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:53.012949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.013242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.013516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.031664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:53.033055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:53.052914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c87df-cc73-4c2e-b6e9-7e804be214fd None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.088097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.088699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.088976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.089245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.089860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.090153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.091500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.096530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 473/974] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:53.097321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.103683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 502/975] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:53.104539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.192434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.213642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82ed11-6aec-4e4f-a031-8fcb1d1fa8a7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.218459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.224578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50120667-06d2-4a15-957c-1ad49e3fb227 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.254241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.254967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.255260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.255534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.261602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.262201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.262480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.263170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.263811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.264129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.265891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.266229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.267922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.275944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 503/976] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:53.276935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:53.285942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:53.286913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:53.288068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:53.288068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:53.288068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:53.288068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 474/977] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:53.292918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.385816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5bd09-edfc-40c6-a2a4-e76ae934c0e7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.397902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcffbe11-e650-412e-803c-1b32401fb143 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.419577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.420209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.420488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.420785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.421516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] RBAC: Adding query filter params (interface=admin, service_id=29e1215ce1f4429ba0c4106dbb1c4042, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.421825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.423782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.428657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 504/978] 10.222.0.78 () {56 vars in 1249 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/endpoints?service_id=29e1215ce1f4429ba0c4106dbb1c4042&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:53.429401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.436597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.437210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.437911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.438200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.439558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.440141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.442190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.443759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.458834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 475/979] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:53.459683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.464834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.539876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f9a91-21a9-40e3-ab97-ddd996d96b42 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.558568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbcef5-db8f-4c0d-85b2-ea69cc5fdcc7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.574867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.575478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.575759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.576068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.576714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.578081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.592036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 505/980] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:53 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:53.592837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.604467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.605107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.605444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.605674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.606976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.609050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.627826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 476/981] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:47:53.708298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.729130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f43f29-5de2-456e-96f0-8ec52a52da40 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.772125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.772801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.773081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.773351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.782331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.784151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:47:53.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:53.797353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:53.799735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 506/982] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:53.804106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:53.861955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:53.908900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9f38c6-748d-4876-9a03-2bbf1c78c8ef None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:53.943610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:53.944270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:53.944541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:53.944816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:53.945534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:53.945845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:53.947267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:53.965016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 477/983] 10.222.0.78 () {56 vars in 1149 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:53.970988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.077204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d22ed5-9aae-4ad9-936f-06223f96be9b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:54.115155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:54.115769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:54.116132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:54.116409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:54.132369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:54.186884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:54.188385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:54.288406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 507/984] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:47:53 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:47:54.293854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.395550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c79782-db84-46df-a6ec-db2a6d4f293c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:54.409262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:54.431960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:54.432592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:54.432873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:54.433147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:54.443370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:54.447160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:47:54.464367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:54.465301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:47:54.471427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:47:54.471427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:47:54.471427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:54.471427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 478/985] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:54.471670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.518887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:54.572946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cb79b-32a1-41b8-ad38-8718318480b3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:54.607559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:54.608214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:54.608500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:54.608776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:54.609424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:54.609731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:54.611300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:54.629005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 508/986] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:47:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:54.634650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.741552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e61131-5bb9-49a5-9a30-ddd7a08f58ab None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:54.776641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:54.777275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:54.777555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:54.777834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:54.784418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:54.785851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:47:54.797724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:54.799169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:47:54.800161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:54.805708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 479/987] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:54.806318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.906386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58313f-a52b-49b3-b5f9-91292c3d8f8a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:54.913869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:54.942696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:54.943302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:54.943584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:54.943886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:54.944578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:54.944886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:54.946320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25738c08-a986-4799-8bb3-8185c90a5fd6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:54.963171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 509/988] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:47:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:54.964770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:54.992873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:55.072890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b1c9e-dd06-418a-bf49-6491bb2846d9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:55.107186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:55.107897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:55.108182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:55.108458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:55.109186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] RBAC: Adding query filter params (role.id=a3a0c05005d2415ab6cfe90f7a7f7792, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b5793768f3fe46ab8b16908d5f570c91) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:55.109495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:55.110924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e140b-4d4d-4b56-99e0-2401ffccb113 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:55.120180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 480/989] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:47:54 2022] GET /identity/v3/role_assignments?user.id=b5793768f3fe46ab8b16908d5f570c91&scope.project.id=35e2575a816145b297430e41f586390d&role.id=a3a0c05005d2415ab6cfe90f7a7f7792 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:55.573336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180851f-fc17-4894-903a-1b84e6dd53d8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:55.573681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180851f-fc17-4894-903a-1b84e6dd53d8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:55.573890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180851f-fc17-4894-903a-1b84e6dd53d8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:55.575528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 510/990] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:55.580808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:55.581099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:55.581368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:55.593243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e97f9-d0b4-492c-9ef1-5a1852df5330 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:55.593537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e97f9-d0b4-492c-9ef1-5a1852df5330 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:55.593805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e97f9-d0b4-492c-9ef1-5a1852df5330 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:55.595153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 511/991] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:55.600109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:55.600561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:55.600849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:55.601126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:55.601308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:55.619297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:55.620515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:55.675053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 481/992] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:55.696414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 512/993] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:55 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:47:55.949009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:55.962253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:55.969048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.061709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.070304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb23b9de-fcf3-4fec-92e7-89789f483838 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.070852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b96e434-eb9f-41f0-a488-6bc8a87b8514 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.107190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.107827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.108106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.108437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.109242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.110380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.116985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.117622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.117900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.118170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.118752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:56.119589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:56.120584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 09:47:56.122135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:56.122919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.122992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 482/994] 10.222.0.78 () {56 vars in 1126 bytes} [Wed Aug 24 09:47:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:56.126451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:56.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:56.132823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:56.134977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:56.137406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 513/995] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:56.138043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.220318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.244571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa56f19-32dc-4066-9baf-2eab78b44702 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.250787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c5d74a-9c90-454d-a2ba-0e0b351d8060 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.281958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.282582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.282862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.283141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.284455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:56.284763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.286165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.287935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.288546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.288826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.289094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.289720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:56.290023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.291382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.295908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 483/996] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:56.297394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.300647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 514/997] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:56.306064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.333517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.405428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06492f2b-bae0-4139-90c2-a2480cd2e05e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.412917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab7448-53b8-467d-8fd7-d52d7e1d38a1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.439665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.440901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.441439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.441712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.442330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:56.442627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.444013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.453820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 484/998] 10.222.0.78 () {56 vars in 1122 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:56.454527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.456043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.456846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.457248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.457615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.458320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:56.458724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.460137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.465450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 515/999] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:56.471088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.473243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.567570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:56.574374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df1d65-43eb-41f6-b105-80d3df3fd00a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.581461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02fd1ea-9cec-4bc7-9988-2e2a4e7bfd2c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.611469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.612706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.612980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.613249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.613908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.615273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c3c5b-22d2-43cd-8dd2-695d761c4062 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.620336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.621429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.621429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.621653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.622268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] RBAC: Adding query filter params (interface=public, service_id=458f47f109d74e9c9aa95821fc7be460, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:56.622619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.624134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.630301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 516/1000] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/endpoints?service_id=458f47f109d74e9c9aa95821fc7be460&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:56.635311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:56.635810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 485/1001] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:56.761054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d741c1-72dc-4d9e-bdb3-60aaee612f86 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:56.806601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:56.807209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:56.807495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:56.807767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:56.808396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:56.809765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff18e5-ee26-498b-b6b6-8d22c5344aad None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:56.817173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 517/1002] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:47:56 2022] GET /identity/v3/services => generated 1727 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:47:57.097359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd320d5-7cba-40c5-bfea-f413127dcee7 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:57.097865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd320d5-7cba-40c5-bfea-f413127dcee7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:57.097865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd320d5-7cba-40c5-bfea-f413127dcee7 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:57.099444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 486/1003] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:57.104964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:57.105523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:57.105523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:57.124330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:57.125515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:57.134282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:57.198698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 518/1004] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:57 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:57.466621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd320d5-7cba-40c5-bfea-f413127dcee7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:57.543477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:57.588399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd320d5-7cba-40c5-bfea-f413127dcee7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:57.625864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:57.626503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:57.626781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:57.627051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] PATH_INFO: `/v3/services/image` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:57.627631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:57.629391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:57.638586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:57.639599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:47:57.640536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 487/1005] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 24 09:47:57 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:47:57.648011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:57.755903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771d880-0553-4f93-8806-4eb45e2f1d53 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:57.793516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:57.794145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:57.794403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:57.794670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:57.795295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:57.795596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:57.796993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babf9e00-45b5-43f9-b702-5eeebb17c8f2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:57.807024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 519/1006] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:57.808223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:57.914899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a131e67f-5955-4995-b232-5f0aa3669dd6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:57.950771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:57.951399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:57.951675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:57.951975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:57.952619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] RBAC: Adding query filter params (type=image) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:57.952919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:57.954338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2075fa0-7fee-404a-b90d-dbe43c4a55be None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:57.955935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:57.968053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 488/1007] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:58.619318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc0a-3f87-427b-94a0-34a9aa834387 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:58.619985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc0a-3f87-427b-94a0-34a9aa834387 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:58.619985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785acc0a-3f87-427b-94a0-34a9aa834387 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:58.621386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 520/1008] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:58.626526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafee6e-45a5-498d-90ef-e13c281a2d34 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:58.626901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafee6e-45a5-498d-90ef-e13c281a2d34 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:58.627084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aafee6e-45a5-498d-90ef-e13c281a2d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:58.645755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5aafee6e-45a5-498d-90ef-e13c281a2d34 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:58.646967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5aafee6e-45a5-498d-90ef-e13c281a2d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:58.717973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 489/1009] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:58.813215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77e7e-dcff-4a56-8629-2aa157c34724 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:58.813558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77e7e-dcff-4a56-8629-2aa157c34724 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:58.813781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c77e7e-dcff-4a56-8629-2aa157c34724 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:58.815273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 521/1010] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:58.820798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:58.821093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:58.821374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:58.839314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:58.840547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:58.908857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 490/1011] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:47:58.996378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77e7e-dcff-4a56-8629-2aa157c34724 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.006094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:59.104977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c77e7e-dcff-4a56-8629-2aa157c34724 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.138910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.139536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.139836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.140113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.141229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.142650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.144438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 522/1012] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:47:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:59.150114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.186119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.212419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:59.259064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e97bd37-0cff-479a-9d0d-1e1f86543c6e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.272928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:59.296023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.296773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.297054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.297381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.298061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.299632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.303770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f8c563-1709-4558-b06d-ce0f126440f6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.329260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 491/1013] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:47:59 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:47:59.344773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.345434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.345715 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.345989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.346585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.348750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:47:59.358676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:47:59.359762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:47:59.365155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 523/1014] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:47:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:47:59.366483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.488235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa0932-774e-49f0-b55d-c4fd8cb8816f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.499572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:59.527596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.528167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.528457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.528738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.529372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:59.529676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.531109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.536541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 492/1015] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:47:59.542485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.649838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c971225-fc0a-4381-8f42-beefcc7a415f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.684680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.685350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.685637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.685909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.686538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:47:59.686837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.688225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349b1c4-640f-40f5-a739-16abd23a71e9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.698176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 524/1016] 10.222.0.78 () {56 vars in 1112 bytes} [Wed Aug 24 09:47:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:47:59.699924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:47:59.805127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aa2f13-06a7-438a-ab12-3db90235418f None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:47:59.841203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:47:59.841816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.842098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.842376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.843039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:47:59.844462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:47:59.861743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbf3a-a52f-4807-973c-bea9edcf072c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.862113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbf3a-a52f-4807-973c-bea9edcf072c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.862473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbbf3a-a52f-4807-973c-bea9edcf072c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.864104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 525/1017] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:47:59.866681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 493/1018] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:47:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:47:59.873546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fdf2-788d-436a-bce2-5bf3102992c2 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:47:59.873892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fdf2-788d-436a-bce2-5bf3102992c2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:47:59.874286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b55fdf2-788d-436a-bce2-5bf3102992c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:47:59.881461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:47:59.896266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9b55fdf2-788d-436a-bce2-5bf3102992c2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:47:59.897637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9b55fdf2-788d-436a-bce2-5bf3102992c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:47:59.979722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 526/1019] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:47:59 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:00.249082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:00.279454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:00.355754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f857b-dd7e-41e6-9f63-2cc660c00051 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:00.392978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:00.393595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:00.393872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:00.394207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:00.394915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:00.396314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f20bd-2eaf-4084-b7f9-1bed9aae1bdf None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:00.416436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 494/1020] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:48:00 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:01.339742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66656-db34-4f5d-b605-191aa2b0b9dc None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.340286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66656-db34-4f5d-b605-191aa2b0b9dc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.340558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea66656-db34-4f5d-b605-191aa2b0b9dc None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.342151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 527/1021] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:01.349401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.349897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.349995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.370277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:01.371490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:01.467328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 495/1022] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:01.761896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66656-db34-4f5d-b605-191aa2b0b9dc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:01.868507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea66656-db34-4f5d-b605-191aa2b0b9dc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:01.895948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:01.905328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:01.905963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.906244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.906516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.907616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:01.909089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2184aa5c-4586-40a7-b94c-e874359cdcce None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:01.911817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 528/1023] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:01.917269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:01.935700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:01.951337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df3990-b1ba-4c87-b4b4-9a3be2b019c8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.952277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df3990-b1ba-4c87-b4b4-9a3be2b019c8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.952575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df3990-b1ba-4c87-b4b4-9a3be2b019c8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.954081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 529/1024] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:01.958841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:01.959121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:01.959392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:01.977643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:01.978842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:02.026774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada65cd4-f8b1-4a7d-8c13-5b0ae91ecb63 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.050956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 530/1025] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:02.062924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.063535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.063834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.064111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.064767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:02.066194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1c718-c38d-40a3-b67d-36be0b37316d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:02.089936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 496/1026] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:48:01 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:02.322184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:02.425017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c14373-aaa8-4c17-8dbc-65211c279b63 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.447117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b75d-4e78-4f76-8c3a-b7434816ff85 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.447117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b75d-4e78-4f76-8c3a-b7434816ff85 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.447117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30b75d-4e78-4f76-8c3a-b7434816ff85 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.448243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 497/1027] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:48:02.453384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.453737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.454084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.458624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:02.468138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.468782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.469064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.469334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.474060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:02.475390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:02.480239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:02.485268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:02.486412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:02.487299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 531/1028] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:02.492372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:02.558210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 498/1029] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:02.600164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c778b-eceb-44a5-a544-e86532c374fb None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.635398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.636086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.636365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.636636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.637293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:02.637593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:02.638949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:02.648041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 532/1030] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:02.653504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:02.656211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:02.760070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2fa3f-9a0c-459a-9722-79173b19202c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.795171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.795810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.796113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.796382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.807671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:02.809417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:02.824436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:02.825410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:02.827033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:02.827033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:02.827033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:02.827472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 499/1031] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:02.827661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:02.831111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:02.879817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:02.891000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:02.933826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caef3bb-39dd-45b9-ba9b-e9ba8ad4a36e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.939872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c1cf9e-7944-4aba-a987-84f8d78389ad None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:02.971087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.971956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.972420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.972693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] PATH_INFO: `/v3/services/image` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.973455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:02.975480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:02.980693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:02.981306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:02.981589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:02.981865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:02.982507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:02.982812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:02.984227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:02.984325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:02.985289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:02.986139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:02.990815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 533/1032] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:02.991615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.004448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 500/1033] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:03.004950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.107830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a07eb1-f63e-4438-95a1-5bdac47dda3b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.113603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f4482c-2c6c-4426-a5a0-0e5b68125999 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.121938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.133061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.146987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.147897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.148362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.148811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.149644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] RBAC: Adding query filter params (name=image) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:03.150096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.151612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.155423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.156115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.156394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.156660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.162948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 534/1034] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:03.164035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.165534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.166976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:48:03.178910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:03.179921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:03.185539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 501/1035] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:03.186158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.283699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb34ae7-728a-4110-93f1-c238e7c4b77a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.294665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed66f-4397-42ec-9976-a1b5a196c9af None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.321520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.322236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.322514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.322790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.323416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:03.323716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.325378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.329696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.330301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.330602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.330870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.331584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:03.331916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.333507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.335729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 535/1036] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:03.336682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.351680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 502/1037] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:03.357472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.388521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.391807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.452530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e1448-d7ad-45c8-a620-ee9346bed0d2 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.475344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1298db66-23a5-41f8-ba8b-f03d84ce17e6 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.489133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.489896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.490185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.490813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.491556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] RBAC: Adding query filter params (interface=public, service_id=08fac1545fc54b08a743e9fcaadd104f, region_id=RegionOne) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:03.491897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.493278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.498887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 536/1038] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/endpoints?service_id=08fac1545fc54b08a743e9fcaadd104f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:03.504661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.514359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.515270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.515773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.516544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.610166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.611631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.625157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b014d2da-4339-47ec-a81a-e8f19cb29e97 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.628712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.652171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:03.667254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.667929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.668192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.668464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.669075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.670691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.678235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 537/1039] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/services => generated 1957 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:48:03.738129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 503/1040] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:48:03.745333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:03.870450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c555cd-1f84-4444-956f-bf0a90319ce7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:03.925731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:03.926288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:03.927331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:03.927331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:03.941081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:03.943134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7d604c53-a4f2-4ae3-9f1c-50564f22323f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:03.959381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:03.960739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:03.961850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:03.961850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:03.961850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:03.961850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 538/1041] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:03.966523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.078607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0bcf75-0921-402a-a5f2-05f76a1f0153 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.108714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.114756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b70c-4b2c-45f3-a785-5ddc1f3165d2 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.115017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.115306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b70c-4b2c-45f3-a785-5ddc1f3165d2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.115606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.115710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6b70c-4b2c-45f3-a785-5ddc1f3165d2 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.115921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.116219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.116865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:04.117225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 539/1042] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:48:04.117328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:04.118647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:04.122023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.122300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.122567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.131025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 504/1043] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:04.136765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.141160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:04.142349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:04.157937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.228802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 540/1044] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:04.247899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cbec52-e130-4be7-b865-c132e6ce78c4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.283448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.284117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.284401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.284677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.291541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:04.293018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:48:04.305036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:04.305915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:04.306876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:04.310945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 505/1045] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:04.315838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.414021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af80f8-d5dc-4dc9-81c2-fa4243c619f3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.454273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.454875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.455138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.455406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.456121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:04.459001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:04.459001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:04.472051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 541/1046] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:04.478521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.501382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.515467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.538689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.592699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d8a589-891b-4e04-bc57-9ad8d70c3788 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.612886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b800d9-ab79-4244-8f41-9a6bfb624db4 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.629261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.629941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.630200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.630648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.631977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] RBAC: Adding query filter params (role.id=a3a0c05005d2415ab6cfe90f7a7f7792, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:04.632548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:04.634220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:04.640190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 506/1047] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=a3a0c05005d2415ab6cfe90f7a7f7792 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:04.658193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.658976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.659332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.659710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.673744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:04.678709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:04.679703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:04.681020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:04.682135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 542/1048] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:04.686019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.804779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f266532-651b-4c34-bf53-c25d83723b76 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:04.824546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.845498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:04.846175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:04.846462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:04.846739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:04.847404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:04.847710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:04.849328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:04.863308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 507/1049] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:04.864238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:04.937213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:04.977708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c658a25-0648-4dcf-9ce6-0f6f3a8eda01 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:05.019059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:05.020169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.020526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.020815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] PATH_INFO: `/v3/users/b51e19c13257452a82a356dc0b05a127` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.036117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] RBAC: Authorizing `identity:get_user(user_id=b51e19c13257452a82a356dc0b05a127)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:05.039005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:05.045605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 543/1050] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:04 2022] GET /identity/v3/users/b51e19c13257452a82a356dc0b05a127 => generated 285 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:05.046844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:05.152252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736bb38-96de-443d-a06b-78f7c2663ee7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:05.186967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:05.187586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.187895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.188171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.234802 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:05.259059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:05.260612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:05.367360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 508/1051] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:05.368592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:05.479819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3169d1-3c34-46fb-bb81-1dff78653772 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:05.506918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:05.515763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:05.516426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.516708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.516981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.518107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:05.519541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036e4f8-b964-49ed-b591-da508d267c92 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:05.526590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 544/1052] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:05 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:05.529780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:05.609437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:05.632980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e837685b-384c-4fcc-ab83-c4a9ffdf3f94 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:05.668168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:05.668525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c80a-0e56-47f4-877d-e6698c71a100 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.668740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c80a-0e56-47f4-877d-e6698c71a100 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.668850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.669022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe84c80a-0e56-47f4-877d-e6698c71a100 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.669118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.669309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.670038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=9e399f9c62964f039b2f08b51a737c35, user.id=b51e19c13257452a82a356dc0b05a127) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:05.670349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:05.670757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 545/1053] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:05.671783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:05.675431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:05.675714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:05.676030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:05.684099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 509/1054] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:05 2022] GET /identity/v3/role_assignments?user.id=b51e19c13257452a82a356dc0b05a127&scope.project.id=9e399f9c62964f039b2f08b51a737c35&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:05.695898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:05.697127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:05.773067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 546/1055] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:06.040191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:06.147715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef2178-8f30-45e0-bbc0-805d1e887de2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:06.182318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:06.182953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.183232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.183505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] PATH_INFO: `/v3/services/image` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.184108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:06.185469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:06.194227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:06.195506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:06.196829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 510/1056] 10.222.0.78 () {56 vars in 1105 bytes} [Wed Aug 24 09:48:06 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:06.204935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:06.311140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549d5bd-57ea-4449-8f4a-84e80e7277e4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:06.343590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:06.353843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:06.354454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.354728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.354997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.355683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] RBAC: Adding query filter params (name=image) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:06.356016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:06.357380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:06.367978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 547/1057] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:06.369248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:06.379274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:06.483388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064ae7f-f0b7-4b44-94ed-e5c0c46e076c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:06.518566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:06.519169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.519444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.519714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.520360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] RBAC: Adding query filter params (type=image) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:06.520656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:06.523141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872c402c-ee87-437b-85d6-c3ebedc08e0d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:06.528321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 511/1058] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:06.533787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:06.557377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d146814-5691-4692-9655-aaa6a8d717e7 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.557875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d146814-5691-4692-9655-aaa6a8d717e7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.557953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d146814-5691-4692-9655-aaa6a8d717e7 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.559422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 512/1059] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:48:06.564152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.564433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.564774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.583154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:06.584932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:06.643203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cafb7-381d-4a88-8f64-d301b95329de None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:06.649856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:06.659871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 513/1060] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:06.686710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:06.687322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:06.687597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:06.687891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:06.688568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:06.690176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:06.711430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 548/1061] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:48:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:06.929725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.033600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d076ee-b509-4b4d-a750-0a818c78f074 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.075376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.076035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.076315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.076582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.087392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:07.092121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:07.093110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:07.094344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:07.095872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 514/1062] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:48:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:07.099349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.148048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:07.207875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af0f048-828d-4ca9-8c59-4889d2153a1e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.243234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.243876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.244166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.244437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.245106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:07.245412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:07.246785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:07.255236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 549/1063] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:07.260934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.321800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:07.368124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bddcc1-8eff-41bc-8dfa-b08d37dcfb24 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.402860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.403867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.404151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.404426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.415258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:07.416742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:07.431869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:07.432812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:07.438932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:07.438932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:07.438932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:07.438932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 515/1064] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:07.439187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.546615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf295c7-ffc5-40e2-a001-cad8f337189c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.556423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:07.583606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.584357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.584766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.585148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.585802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:07.586110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:07.587482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca21cc-8af8-4b33-98d6-fb3649274655 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:07.604472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 550/1065] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:07.605497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.628366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee941-555d-4d93-b49e-1bb17f12b3e8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.628662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee941-555d-4d93-b49e-1bb17f12b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.628936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbee941-555d-4d93-b49e-1bb17f12b3e8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.630294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 551/1066] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:48:07.635771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.636086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.636362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.655662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:07.656890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:07.714144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982ed24-fb72-4f11-a26f-336609bc2a7a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.726604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:07.731969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 552/1067] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:07.757677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.758324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.758609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.758882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.767608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:07.769093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:48:07.780903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:07.781839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:07.783373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 516/1068] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:07.787823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:07.890426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:07.894850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f1153-7078-4762-a127-e05cd277129b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:07.933570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:07.934470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:07.934933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:07.935465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:07.936454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:07.936946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:07.938540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:07.963943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 553/1069] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:07.965381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.001632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.081838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a091f44-005e-4eb7-af82-82dc117313c5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.120118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c41b7-0ba8-4fc7-814c-693d49905d05 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.122471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.123110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.123386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.123654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.140034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.141591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.160909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:08.161317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:08.162225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:08.163998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:08.163998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:08.163998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:08.164174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.164174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.164174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.164381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 517/1070] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:08.168612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.177651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:08.182976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:08.184016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:08.185318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 554/1071] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:08.189985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.201518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.210374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.306809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bee538d-e986-40b8-9e54-e2e4e8fdfc30 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.314652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a92a85-6f47-4a5c-a601-06566a8fd653 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.346515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.347406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.347677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.348349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:08.348646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.350029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.357595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.358459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.358813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.359088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.359750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:08.360445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.361491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.368166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 518/1072] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:08.371034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 555/1073] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:08.371441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.378734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.484246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.486358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.510554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2fb9ad-8c07-4248-9fbe-1425786412bd None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.515552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa288abf-ef2c-45e0-8bc8-ab381c2f944d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.554993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.555649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.555966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.556240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.560089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.560674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.560951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.561227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] PATH_INFO: `/v3/users/b51e19c13257452a82a356dc0b05a127` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.562405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=b51e19c13257452a82a356dc0b05a127)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.563742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.563935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.565222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.572100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 556/1074] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/users/b51e19c13257452a82a356dc0b05a127 => generated 285 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:08.573116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:48:08.584264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:08.586122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:08.587037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 519/1075] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:08.592027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.695156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09455eb-29b2-4b87-8899-c713ef158ee2 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.703828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d96ef-6664-4d05-80ba-3baccf5c88f0 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.731711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.732400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.732690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.732962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.734122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.735583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.742794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 557/1076] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:08.743974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.747600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.748355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.748648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.748928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.749674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:08.749994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.751613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b69390-c589-45fb-b64c-80159932a462 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.764613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.764818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:08.775976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 520/1077] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:08.778228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abb9f-d954-413a-99ac-e9128d36e486 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.778543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abb9f-d954-413a-99ac-e9128d36e486 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.778823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738abb9f-d954-413a-99ac-e9128d36e486 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.781144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 521/1078] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:08.781726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738abb9f-d954-413a-99ac-e9128d36e486 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:08.865278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59d95c7-b993-4224-b646-3b427018e9e9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.903173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738abb9f-d954-413a-99ac-e9128d36e486 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:08.907806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.908510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.908794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.909156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35/users/b51e19c13257452a82a356dc0b05a127/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.911269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e399f9c62964f039b2f08b51a737c35, user_id=b51e19c13257452a82a356dc0b05a127, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.913477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59114ad-611d-4fa3-86c0-cf9d67c538b9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.928006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 558/1079] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:48:08 2022] PUT /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35/users/b51e19c13257452a82a356dc0b05a127/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:48:08.937341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.937735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.937992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.943755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:08.945320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:08.945617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:08.946237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/a3a0c05005d2415ab6cfe90f7a7f7792` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:08.948757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=7f645f811e804835a793f372a1ac7111, role_id=a3a0c05005d2415ab6cfe90f7a7f7792)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:08.950518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:08.960782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:08.962074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:08.966464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 522/1080] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:48:08 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/7f645f811e804835a793f372a1ac7111/roles/a3a0c05005d2415ab6cfe90f7a7f7792 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:48:09.071875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 559/1081] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:09.399536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:09.468168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:09.517299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf1c0d7-7e56-4308-acc3-b40cd3786b38 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:09.556102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:09.556718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:09.557031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:09.557281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:09.628577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:09.630050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:09.696713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:09.749147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 523/1082] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:09.754993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:09.758873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:09.868389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2829c8-a0dc-4155-acd5-c77b72ca22d9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:09.902567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:09.903193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:09.903472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:09.903742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:09.914657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:09.916149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-6a517a74-ddba-40f2-aa91-9b14041879f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:09.931439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:09.932611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:09.933612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:09.933612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:09.933612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:09.933612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 560/1083] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:09.938499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:10.041091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d2af1-549f-4746-b97e-ad48f9013410 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:10.077306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:10.077925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:10.078208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:10.078485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:10.079130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:10.079437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:10.080853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa100c-985e-4b84-b48c-11c1c2d962f2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:10.093451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 524/1084] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:11.010468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509592-aefe-4c92-bd6d-c1f631228813 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.010997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509592-aefe-4c92-bd6d-c1f631228813 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.011078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509592-aefe-4c92-bd6d-c1f631228813 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.012450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 561/1085] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:11.014563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265c47-63d3-4f30-a70d-5b0fc4aadfa8 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.014966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265c47-63d3-4f30-a70d-5b0fc4aadfa8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.015103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f265c47-63d3-4f30-a70d-5b0fc4aadfa8 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.016468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 525/1086] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:11.017541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.017824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.018095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.021318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.021602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.021873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.036612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:11.037800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:11.040217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:11.041444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:11.055893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.089129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.106054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 562/1087] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:11.110336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 526/1088] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:11.382276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.424941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.488060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6bae5-5423-4b5d-8a7d-c29ae813b733 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.522250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.522905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.523872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.524111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.534649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:11.538063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59c13a-0f33-4d32-8270-22876e659e56 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:11.539295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:11.540573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:11.541448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 563/1089] 10.222.0.78 () {56 vars in 1120 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:11.546414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.576501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.577150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.577431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.577703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.588611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:11.593333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:11.594261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:11.595721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 527/1090] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:11.597054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.600625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.613692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.649662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9885-3fcc-4f50-875e-92bfbaa721f0 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.684270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.684873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.685155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.685426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.686116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:11.686470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:11.687876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:11.695857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 564/1091] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:11.701889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.704238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d70218-96de-41ff-a59c-2e0e4f6b7f11 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.740086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.740692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.740968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.741238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.742509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:11.742843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:11.744294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:11.752699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 528/1092] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:11.758853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.809979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6720fc01-4abc-43f1-b5ac-0af198e3c597 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.819477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.831515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:11.845977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507331-ed15-4512-a733-076b5587820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.846610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507331-ed15-4512-a733-076b5587820b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.846895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507331-ed15-4512-a733-076b5587820b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.847170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11507331-ed15-4512-a733-076b5587820b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.858718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11507331-ed15-4512-a733-076b5587820b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:11.860216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11507331-ed15-4512-a733-076b5587820b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:11.863238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f90be7-c83f-46d7-8093-8d6a41535c53 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-11507331-ed15-4512-a733-076b5587820b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:11.874465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:11.875465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:11.876451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:11.876451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:11.876451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:11.876451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 565/1093] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:48:11.881373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507331-ed15-4512-a733-076b5587820b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.902183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:11.902727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:11.903008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:11.903290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] PATH_INFO: `/v3/users/b51e19c13257452a82a356dc0b05a127` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:11.904457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=b51e19c13257452a82a356dc0b05a127)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:11.905891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:11.907769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 529/1094] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/users/b51e19c13257452a82a356dc0b05a127 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:11.914436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:11.985147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11507331-ed15-4512-a733-076b5587820b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.016427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f33a0-df5c-4618-813f-a9e816855dc5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.019743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.020376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.020659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.020929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.021574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:12.021878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.023272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.035726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 566/1095] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:12.041529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.049990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.050634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.050916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.051189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.062550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.063935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.122965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.125703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.145063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fc2f4-7952-48ea-93f3-097f05383cce None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.182990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.183536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.183841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.184117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.192548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.194003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:48:12.207007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:12.208266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:12.209247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 567/1096] 10.222.0.78 () {56 vars in 1093 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:12.214710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.230276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 530/1097] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:48:12.236102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.278193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.296761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.315924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7973c5-a2c2-4e80-bf12-679ba2974427 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.337171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ee733-6698-4ce9-8468-6c7133515237 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.351578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.352332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.352610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.353033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.354279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:12.354606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.356151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.373738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.374431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.374712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.374982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.376125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.376443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 568/1098] 10.222.0.78 () {56 vars in 1133 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:12.377693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.379447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 531/1099] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:12.382095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.385698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.489558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715da918-c72a-4f62-b327-58a44a7f2854 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.494050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccb1b3f-537a-4a1f-818d-29e54d473f74 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.503583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.526359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754410-8445-4cf1-9042-4079a712987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.527109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754410-8445-4cf1-9042-4079a712987c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.527391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754410-8445-4cf1-9042-4079a712987c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.527667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21754410-8445-4cf1-9042-4079a712987c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.528653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.529258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.529535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.529807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.530528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=9e399f9c62964f039b2f08b51a737c35, user.id=b51e19c13257452a82a356dc0b05a127) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:12.530830 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.532304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.536148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:12.538399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 532/1100] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/role_assignments?user.id=b51e19c13257452a82a356dc0b05a127&scope.project.id=9e399f9c62964f039b2f08b51a737c35&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:12.606103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754410-8445-4cf1-9042-4079a712987c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.607748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21754410-8445-4cf1-9042-4079a712987c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.721194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 569/1101] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:48:12.727678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.834500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2061d-6aba-4d7c-951e-b7f6afabefc5 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:12.874049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:12.874690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:12.874972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:12.875247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:12.887001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:12.888500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:12.902592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:12.903739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:48:12.905081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:12.905081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:48:12.905081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:12.907942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 533/1102] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:12.909415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754410-8445-4cf1-9042-4079a712987c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:12.950136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:13.014395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21754410-8445-4cf1-9042-4079a712987c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:13.053847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:13.054541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:13.055716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:13.056007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:13.056648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:13.056949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:13.058316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:13.069753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 570/1103] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:13.076022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:13.102529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:13.181079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408844d-a768-4f75-80c9-98f6309fe5c2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:13.216432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:13.217072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:13.217354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:13.217627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:13.224645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:13.226093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:48:13.237437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:13.238431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:13.239366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 534/1104] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:13.244516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:13.342083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:13.345912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8cd3d-af0b-4c3a-a84d-398f9dea6bc5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:13.384512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:13.385347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:13.385642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:13.385914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:13.386613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:13.386919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:13.388379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1e690-34c6-40fe-9a63-9acdac349130 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:13.401474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 571/1105] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:48:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:13.407563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:13.499591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:13.515247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b648227-7954-4b3e-8b25-e99bef8a4cf8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:13.548952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:13.549552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:13.549829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:13.550098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:13.550812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] RBAC: Adding query filter params (role.id=a3a0c05005d2415ab6cfe90f7a7f7792, scope.project.id=35e2575a816145b297430e41f586390d, user.id=7f645f811e804835a793f372a1ac7111) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:13.551118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:13.552598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e962333b-e0be-4d8e-843f-f7d37450d265 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:13.558941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 535/1106] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:13 2022] GET /identity/v3/role_assignments?user.id=7f645f811e804835a793f372a1ac7111&scope.project.id=35e2575a816145b297430e41f586390d&role.id=a3a0c05005d2415ab6cfe90f7a7f7792 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:14.556966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4771789-84a5-45b8-9f40-37e8fbc47f20 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:14.557497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4771789-84a5-45b8-9f40-37e8fbc47f20 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:14.557497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4771789-84a5-45b8-9f40-37e8fbc47f20 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:14.558885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 572/1107] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:14.565447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:14.565987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:14.565987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:14.586894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:14.588187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:14.656778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 536/1108] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:14.944094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4771789-84a5-45b8-9f40-37e8fbc47f20 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:15.050288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4771789-84a5-45b8-9f40-37e8fbc47f20 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:15.083838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:15.084448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:15.084724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:15.084991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:15.086101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:15.087679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4031d6-9099-4473-b6b1-e539bb045507 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:15.089247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 573/1109] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:48:15.094960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:15.179360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:15.203725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14507e-a950-446c-bdab-17bab5d50e03 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:15.236996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:15.237592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:15.237867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:15.238141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:15.238774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:15.240270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824b3dd7-3e4d-448f-9b8e-96840c1b8167 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:15.263754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 537/1110] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:48:15 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:17.533824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff147-131f-43c1-97a1-c268d7348567 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:17.534417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff147-131f-43c1-97a1-c268d7348567 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:17.534417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ff147-131f-43c1-97a1-c268d7348567 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:17.535741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 574/1111] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:17.541390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:17.541811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:17.541946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:17.559893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:17.561101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:17.627453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 538/1112] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:17.901268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ff147-131f-43c1-97a1-c268d7348567 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:17.928769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:18.004908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ff147-131f-43c1-97a1-c268d7348567 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:18.039268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:18.039932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:18.040275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:18.040604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] PATH_INFO: `/v3/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:18.042041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] RBAC: Authorizing `identity:get_role(role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:18.043423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:18.044798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 575/1113] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:17 2022] GET /identity/v3/roles/00b090a41bf543a3a320c3787e8a836d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:18.050645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:18.113512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:18.153781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9bfb7-cbc2-466a-a3ce-13f0541862d7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:18.188968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:18.189599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:18.189877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:18.190149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] PATH_INFO: `/v3/users/70418a819b1049dd9a7396b6b88f0476` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:18.203715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] RBAC: Authorizing `identity:get_user(user_id=70418a819b1049dd9a7396b6b88f0476)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:18.205191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:18.206833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 539/1114] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:18 2022] GET /identity/v3/users/70418a819b1049dd9a7396b6b88f0476 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:18.212956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:18.304952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:18.312536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51e621-a8a5-4d6a-8859-dcdcfea1ef22 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:18.345961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:18.346552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:18.346826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:18.347094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:18.420487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:18.421846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:18.521063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:18.530261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 576/1115] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:18.536516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:18.636088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15271ae4-230c-46b9-8f1f-e8adc6133386 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:18.661531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:18.675253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:18.675872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:18.676158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:18.676433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:18.677622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:18.679057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba4485-5090-4a6c-9a80-c1c61402c0f0 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:18.680751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 540/1116] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:18 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:18.687464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:18.796596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e434-7ecf-4076-8805-adc1a36ad02b None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:18.834309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:18.834907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:18.835182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:18.835451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:18.836207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=9e399f9c62964f039b2f08b51a737c35, user.id=70418a819b1049dd9a7396b6b88f0476) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:18.836509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:18.838979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44337691-4f2c-4568-9bb2-f32e66b5a756 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:18.844640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 577/1117] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:18 2022] GET /identity/v3/role_assignments?user.id=70418a819b1049dd9a7396b6b88f0476&scope.project.id=9e399f9c62964f039b2f08b51a737c35&role.id=00b090a41bf543a3a320c3787e8a836d => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:20.763315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa795-7bb3-4608-9eed-73dd9862c4ca None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:20.763838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa795-7bb3-4608-9eed-73dd9862c4ca None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:20.763838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35fa795-7bb3-4608-9eed-73dd9862c4ca None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:20.765158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 541/1118] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:20.770477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:20.770865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:20.771020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:20.789556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:20.790745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:20.845492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:20.860609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 578/1119] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:21.134027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fa795-7bb3-4608-9eed-73dd9862c4ca None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:21.238113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35fa795-7bb3-4608-9eed-73dd9862c4ca None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:21.272351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:21.272979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:21.273261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:21.273534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] PATH_INFO: `/v3/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:21.274929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] RBAC: Authorizing `identity:get_role(role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:21.276529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:21.277859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 542/1120] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:21 2022] GET /identity/v3/roles/00b090a41bf543a3a320c3787e8a836d => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:21.283722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:21.384859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952e4a1-3032-4db6-9f55-20d28f18ba50 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:21.419961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:21.420581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:21.420858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:21.421128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] PATH_INFO: `/v3/users/70418a819b1049dd9a7396b6b88f0476` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:21.422241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] RBAC: Authorizing `identity:get_user(user_id=70418a819b1049dd9a7396b6b88f0476)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:21.423679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:21.425290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 579/1121] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:21 2022] GET /identity/v3/users/70418a819b1049dd9a7396b6b88f0476 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:21.431119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:21.495181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:21.547858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7f9a2-35bd-4c3c-a019-6b923d673105 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:21.582253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:21.582874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:21.583150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:21.583417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:21.584535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:21.585957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81957e1-55e2-49cb-87fb-dcee9fc2ec1d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:21.587636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 543/1122] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:21 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:21.594679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:21.649826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:21.693664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4c5e07-4402-41e2-a047-ad1be4859701 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:21.728108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:21.729618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:21.729901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:21.730172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35/users/70418a819b1049dd9a7396b6b88f0476/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:21.732334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9e399f9c62964f039b2f08b51a737c35, user_id=70418a819b1049dd9a7396b6b88f0476, role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:21.733765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b7e3ac-0116-4801-b976-29c3480621a8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:21.746879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 580/1123] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:48:21 2022] PUT /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35/users/70418a819b1049dd9a7396b6b88f0476/roles/00b090a41bf543a3a320c3787e8a836d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:48:23.669061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb941-06d1-47ca-979d-2ca2481dcc6f None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:23.669603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb941-06d1-47ca-979d-2ca2481dcc6f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:23.669603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bb941-06d1-47ca-979d-2ca2481dcc6f None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:23.670954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 544/1124] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:23.676325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:23.676703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:23.676874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:23.694590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:23.695799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:23.762481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 581/1125] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:24.035254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb941-06d1-47ca-979d-2ca2481dcc6f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:24.137512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bb941-06d1-47ca-979d-2ca2481dcc6f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:24.162951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:24.172881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:24.173518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:24.173794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:24.174064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] PATH_INFO: `/v3/roles/00b090a41bf543a3a320c3787e8a836d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:24.178787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] RBAC: Authorizing `identity:get_role(role_id=00b090a41bf543a3a320c3787e8a836d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:24.180244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:24.181577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 545/1126] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:24 2022] GET /identity/v3/roles/00b090a41bf543a3a320c3787e8a836d => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:24.187632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:24.233291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:24.286427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ad128-36c2-4179-9f08-3f1e3c00a2d2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:24.320410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:24.321036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:24.321313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:24.321584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] PATH_INFO: `/v3/users/70418a819b1049dd9a7396b6b88f0476` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:24.322694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] RBAC: Authorizing `identity:get_user(user_id=70418a819b1049dd9a7396b6b88f0476)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:24.324238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:24.326158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 582/1127] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:24 2022] GET /identity/v3/users/70418a819b1049dd9a7396b6b88f0476 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:24.332444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:24.431145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796650a-3f69-43c7-9c31-c83559467a17 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:24.465135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:24.465743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:24.466024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:24.466300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:24.527638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:24.535700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:24.537236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:24.634659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 546/1128] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:24.641379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:24.739885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24431cd6-fc4b-43bf-9e2e-7f8b28668c9e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:24.758769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:24.779910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:24.780539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:24.780823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:24.781102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] PATH_INFO: `/v3/projects/9e399f9c62964f039b2f08b51a737c35` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:24.782224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=9e399f9c62964f039b2f08b51a737c35)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:24.783653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc78d54-5e0a-4f53-896b-2b4c684fc1f0 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:24.785406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 583/1129] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:24 2022] GET /identity/v3/projects/9e399f9c62964f039b2f08b51a737c35 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:24.791860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:24.890523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:24.899250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f4bd-2ea0-4992-8492-48c1db9e6628 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:24.939729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:24.940377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:24.940857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:24.941325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:24.942394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] RBAC: Adding query filter params (role.id=00b090a41bf543a3a320c3787e8a836d, scope.project.id=9e399f9c62964f039b2f08b51a737c35, user.id=70418a819b1049dd9a7396b6b88f0476) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:24.943717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:24.946738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a69172-71fa-4c3d-b10c-c79bd68f6a99 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:24.958262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 547/1130] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:24 2022] GET /identity/v3/role_assignments?user.id=70418a819b1049dd9a7396b6b88f0476&scope.project.id=9e399f9c62964f039b2f08b51a737c35&role.id=00b090a41bf543a3a320c3787e8a836d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:27.000457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195bb76-081a-4af4-b188-4c4d9aa0a3df None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:27.001028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195bb76-081a-4af4-b188-4c4d9aa0a3df None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:27.001028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195bb76-081a-4af4-b188-4c4d9aa0a3df None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:27.002359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 584/1131] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:27.007879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:27.008311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:27.008423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:27.028803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:27.029992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:27.095277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 548/1132] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:27.363559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195bb76-081a-4af4-b188-4c4d9aa0a3df None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:27.467540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195bb76-081a-4af4-b188-4c4d9aa0a3df None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:27.502119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:27.502750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:27.503032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:27.503305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:27.504413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:27.505849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf8dba-d4e1-401f-9467-889c69be05af None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:27.507435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 585/1133] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:27.513678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:27.600163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:27.613159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cba3cd-a2c3-4443-932a-331702348a06 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:27.646581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:27.647184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:27.647458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:27.647725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:27.648383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:27.649802 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f02033-b949-4ffc-bd0d-99c19011aa41 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:27.677358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 549/1134] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:48:27 2022] POST /identity/v3/projects => generated 311 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:29.657329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f46af-619f-4aa8-b9e3-9d1b003b6829 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:29.657899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f46af-619f-4aa8-b9e3-9d1b003b6829 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:29.657899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f46af-619f-4aa8-b9e3-9d1b003b6829 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:29.659236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 586/1135] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:29.664661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:29.665095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:29.665219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:29.685436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:29.686643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:29.753536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 550/1136] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:30.019391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f46af-619f-4aa8-b9e3-9d1b003b6829 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:30.070842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:30.123035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f46af-619f-4aa8-b9e3-9d1b003b6829 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:30.158091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:30.158711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:30.158988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:30.159256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:30.160371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:30.161795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c03d92-a5e4-4f30-b655-1168ec2a2137 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:30.163356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 587/1137] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:30.169816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:30.236920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:30.269817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681d764-76ee-4405-a509-34461fecfaa9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:30.303297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:30.303976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:30.304257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:30.304530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:30.305178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:30.306609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4dc32c-7c9c-4004-b695-3b92fd88e496 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:30.328186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 551/1138] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:48:30 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:32.268647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd15c35-dae3-4e6d-a03b-7e141a2bc04a None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:32.269214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd15c35-dae3-4e6d-a03b-7e141a2bc04a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:32.269214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd15c35-dae3-4e6d-a03b-7e141a2bc04a None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:32.270521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 588/1139] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:32.275843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:32.276323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:32.276816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:32.297322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:32.298520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:32.366036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 552/1140] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:32.635986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd15c35-dae3-4e6d-a03b-7e141a2bc04a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:32.739998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd15c35-dae3-4e6d-a03b-7e141a2bc04a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:32.757516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:32.774871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:32.775495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:32.775774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:32.776083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:32.786184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:32.790837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:32.792050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:32.792976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 589/1141] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:32.798876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:32.858840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:32.901732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14a7b6-7f68-479c-80a2-ebf2f3e3de7b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:32.936680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:32.937280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:32.937558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:32.937827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:32.938484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:32.938783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:32.940188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:32.948279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 553/1142] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:32.954982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:33.055941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087111-338d-4c6d-a4aa-98db1e5c042e None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:33.097112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:33.097751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:33.098033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:33.098307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] PATH_INFO: `/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:33.111911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] RBAC: Authorizing `identity:get_user(user_id=19365ca2492f4c7ca0ae128ddf6c50fe)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:33.113482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:33.115133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 590/1143] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:32 2022] GET /identity/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:33.121315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:33.222901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c226c0-4860-4ede-bc4b-14d8085e384e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:33.239411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:33.256077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:33.256679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:33.256954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:33.257230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:33.322676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:33.324152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:33.421348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 554/1144] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:33.427457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:33.464143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:33.530396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2e43f-8e6d-4540-91c0-a355d6d8f6c9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:33.564915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:33.565541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:33.565820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:33.566090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] PATH_INFO: `/v3/projects/37abaa8181644c30b20bfb98d1936bbd` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:33.567202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] RBAC: Authorizing `identity:get_project(project_id=37abaa8181644c30b20bfb98d1936bbd)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:33.568667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf6b97-fa25-45f7-8f8b-7767ffea2243 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:33.570309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 591/1145] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:33 2022] GET /identity/v3/projects/37abaa8181644c30b20bfb98d1936bbd => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:33.576731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:33.595561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:33.676845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee11ca7-7ded-48b7-80e0-fc4e7639722c None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:33.710344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:33.710946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:33.711222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:33.711491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:33.712247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=37abaa8181644c30b20bfb98d1936bbd, user.id=19365ca2492f4c7ca0ae128ddf6c50fe) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:33.712553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:33.713987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ae1ebd-3e29-4200-b974-f00be65fda83 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:33.720205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 555/1146] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:33 2022] GET /identity/v3/role_assignments?user.id=19365ca2492f4c7ca0ae128ddf6c50fe&scope.project.id=37abaa8181644c30b20bfb98d1936bbd&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:35.624560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660cb2-0025-48d2-9d4f-c40c2ea754e3 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:35.625085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660cb2-0025-48d2-9d4f-c40c2ea754e3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:35.625177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4660cb2-0025-48d2-9d4f-c40c2ea754e3 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:35.626496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 592/1147] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:48:35.631816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:35.632241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:35.632334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:35.650457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:35.651635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:35.716032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 556/1148] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:48:35.984230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4660cb2-0025-48d2-9d4f-c40c2ea754e3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:36.084082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4660cb2-0025-48d2-9d4f-c40c2ea754e3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:36.086322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:36.116556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:36.117170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:36.117447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:36.117713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:36.128099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:36.132685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:36.133795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:36.134848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 593/1149] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:36.140533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:36.154487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:36.239172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af56fb62-c0d2-4afb-bb00-2778e6fd9449 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:36.273247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:36.273865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:36.274146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:36.274414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:36.275081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:36.275382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:36.276785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:36.286302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 557/1150] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:36.292212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:36.395893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd92d2d-af4b-4e22-a965-1005cc0f0f9a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:36.430519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:36.431149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:36.431430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:36.431701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] PATH_INFO: `/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:36.432869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] RBAC: Authorizing `identity:get_user(user_id=19365ca2492f4c7ca0ae128ddf6c50fe)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:36.434339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:36.436264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 594/1151] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:36 2022] GET /identity/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:36.442163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:36.519079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:36.544344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971079ee-d00b-4a4d-9f1d-7eac3b700971 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:36.580736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:36.581477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:36.581776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:36.582001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] PATH_INFO: `/v3/projects/37abaa8181644c30b20bfb98d1936bbd` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:36.583125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] RBAC: Authorizing `identity:get_project(project_id=37abaa8181644c30b20bfb98d1936bbd)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:36.584593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf941659-d73c-4832-a3d1-8984ef92258a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:36.586247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 558/1152] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:36 2022] GET /identity/v3/projects/37abaa8181644c30b20bfb98d1936bbd => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:36.593191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:36.627528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:36.701979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d3818-28f6-4ba7-b011-17fe40e198bc None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:36.743259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:36.743946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:36.744369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:36.744490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] PATH_INFO: `/v3/projects/37abaa8181644c30b20bfb98d1936bbd/users/19365ca2492f4c7ca0ae128ddf6c50fe/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:36.746371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=37abaa8181644c30b20bfb98d1936bbd, user_id=19365ca2492f4c7ca0ae128ddf6c50fe, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:36.748354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558b190-988a-4b21-905c-0c2c410344a6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:36.768690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 595/1153] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:48:36 2022] PUT /identity/v3/projects/37abaa8181644c30b20bfb98d1936bbd/users/19365ca2492f4c7ca0ae128ddf6c50fe/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:48:38.698162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170fbe4-29ad-4557-ae2a-12c1fa452a1e None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:38.698778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170fbe4-29ad-4557-ae2a-12c1fa452a1e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:38.698778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a170fbe4-29ad-4557-ae2a-12c1fa452a1e None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:38.700125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 559/1154] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:38.706617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109da6-d211-4017-8068-03847436d9c3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:38.707136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109da6-d211-4017-8068-03847436d9c3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:38.707288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109da6-d211-4017-8068-03847436d9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:38.725509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-42109da6-d211-4017-8068-03847436d9c3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:38.726638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-42109da6-d211-4017-8068-03847436d9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:38.793231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 596/1155] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:39.069754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170fbe4-29ad-4557-ae2a-12c1fa452a1e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:39.179428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a170fbe4-29ad-4557-ae2a-12c1fa452a1e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:39.214232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:39.214864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:39.215142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:39.215412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:39.226519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:39.231161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:39.232405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:39.233649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 560/1156] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:39.238376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109da6-d211-4017-8068-03847436d9c3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:39.263026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:39.341519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109da6-d211-4017-8068-03847436d9c3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:39.376030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:39.376638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:39.376919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:39.377192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:39.377859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:39.378162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:39.379537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:39.387496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 597/1157] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:39.394652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:39.398123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:39.499839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b441b6-97d9-41be-9600-a91f8b54bb09 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:39.534396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:39.535015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:39.535292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:39.535561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] PATH_INFO: `/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:39.537294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] RBAC: Authorizing `identity:get_user(user_id=19365ca2492f4c7ca0ae128ddf6c50fe)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:39.538206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:39.539904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 561/1158] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:39 2022] GET /identity/v3/users/19365ca2492f4c7ca0ae128ddf6c50fe => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:39.546275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:39.631775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:39.648363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f77f4e-2f69-4fc2-baf6-58dc78c8e49f None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:39.682314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:39.682921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:39.683201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:39.683476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:39.755425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:39.756905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:39.844565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:39.865355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 598/1159] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:48:39.871525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:39.942889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:39.976417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e1583-0b47-457e-a281-5416eeada908 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:40.010544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:40.011179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:40.011460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:40.011733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] PATH_INFO: `/v3/projects/37abaa8181644c30b20bfb98d1936bbd` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:40.012878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] RBAC: Authorizing `identity:get_project(project_id=37abaa8181644c30b20bfb98d1936bbd)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:40.014455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38f15-d3f4-4bdb-b26e-2f9679e01c55 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:40.016126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 562/1160] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:39 2022] GET /identity/v3/projects/37abaa8181644c30b20bfb98d1936bbd => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:40.022316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:40.119540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f9a73-e683-4ae5-9994-3db1fd8f902a None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:40.155343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:40.155977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:40.156261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:40.156529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:40.157243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=37abaa8181644c30b20bfb98d1936bbd, user.id=19365ca2492f4c7ca0ae128ddf6c50fe) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:40.157549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:40.158970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7319c46-ad27-4d61-8652-883485070cbe None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:40.164730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 599/1161] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:40 2022] GET /identity/v3/role_assignments?user.id=19365ca2492f4c7ca0ae128ddf6c50fe&scope.project.id=37abaa8181644c30b20bfb98d1936bbd&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:42.213922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aa14f-9dac-497a-92d6-e46036de09ad None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:42.214458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aa14f-9dac-497a-92d6-e46036de09ad None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:42.214458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295aa14f-9dac-497a-92d6-e46036de09ad None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:42.215795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 563/1162] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:42.221081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:42.221536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:42.221629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:42.239905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:42.241125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:42.285157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:42.308358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 600/1163] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:42.587852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295aa14f-9dac-497a-92d6-e46036de09ad None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:42.693944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295aa14f-9dac-497a-92d6-e46036de09ad None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:42.728855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:42.729457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:42.729735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:42.730003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:42.739066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:42.799534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:42.801020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:42.906037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 564/1164] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:42.912395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:43.010896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57160a3-4b4d-4e4e-8c23-2146711e778c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:43.048021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:43.048656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:43.048937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:43.049207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:43.058397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:43.059989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-4475fffa-5667-49be-941e-9beb353ba805 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:43.071475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:43.072798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:48:43.073834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 601/1165] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:48:42 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:43.080300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:43.120094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:43.181947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f686282-344a-460b-a5cf-acefe0c95b32 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:43.216578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:43.217258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:43.217447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:43.218435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:43.219102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:43.219409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:43.220873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5ddd2-ed1a-401b-813e-b3c57fad91c7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:43.228724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 565/1166] 10.222.0.78 () {56 vars in 1115 bytes} [Wed Aug 24 09:48:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:45.160777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206e42-dec2-4886-94d5-44524bc0961d None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:45.161319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206e42-dec2-4886-94d5-44524bc0961d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:45.161319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206e42-dec2-4886-94d5-44524bc0961d None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:45.162654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 602/1167] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:45.168034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea524-8a5d-427d-a3af-0b28d4a599fc None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:45.168431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea524-8a5d-427d-a3af-0b28d4a599fc None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:45.168584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ea524-8a5d-427d-a3af-0b28d4a599fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:45.188346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0a1ea524-8a5d-427d-a3af-0b28d4a599fc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:45.189548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0a1ea524-8a5d-427d-a3af-0b28d4a599fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:45.257074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 566/1168] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:45.533084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206e42-dec2-4886-94d5-44524bc0961d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:45.573850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:45.635137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206e42-dec2-4886-94d5-44524bc0961d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:45.668634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:45.669228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:45.669530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:45.669798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:45.670460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:45.672189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b271b39-6092-4379-8d86-32523965d7ca None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:45.688308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 603/1169] 10.222.0.78 () {60 vars in 1138 bytes} [Wed Aug 24 09:48:45 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:47.666414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043cdc-47c2-4c3c-b2c0-093038a38f4f None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:47.667004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043cdc-47c2-4c3c-b2c0-093038a38f4f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:47.667004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043cdc-47c2-4c3c-b2c0-093038a38f4f None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:47.668354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 567/1170] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:47.674493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:47.674966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:47.675069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:47.693256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:47.694444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:47.761462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 604/1171] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:48.025742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043cdc-47c2-4c3c-b2c0-093038a38f4f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:48.057325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:48.127591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043cdc-47c2-4c3c-b2c0-093038a38f4f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:48.162166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:48.162792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:48.163071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:48.163346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] PATH_INFO: `/v3/domains/024daec882c543aca369fb983239b49d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:48.174114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=024daec882c543aca369fb983239b49d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:48.175552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32248f76-0a8f-4a72-acdc-5c9846f3e7de None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:48.177172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 568/1172] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Aug 24 09:48:48 2022] GET /identity/v3/domains/024daec882c543aca369fb983239b49d => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:48.182961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:48.211802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:48.285277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9396fa5-2770-4450-859e-562f156adaaf None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:48.318616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:48.319216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:48.319495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:48.319766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:48.320578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:48.322013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ea974-96c7-49cf-81fb-973dbe61225c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:48.349284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 605/1173] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:48:48 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:50.342894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663776-b708-42b1-83a0-abe451953728 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:50.343463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663776-b708-42b1-83a0-abe451953728 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:50.343463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf663776-b708-42b1-83a0-abe451953728 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:50.345442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 569/1174] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:50.351455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:50.353071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:50.353071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:50.372268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:50.373527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:50.449015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 606/1175] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:50.730220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663776-b708-42b1-83a0-abe451953728 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:50.838119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:50.851666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf663776-b708-42b1-83a0-abe451953728 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:50.892410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:50.893099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:50.893384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:50.893662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] PATH_INFO: `/v3/domains/024daec882c543aca369fb983239b49d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:50.894860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=024daec882c543aca369fb983239b49d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:50.896451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc0abe1-3090-42fc-998a-54ae03615bc1 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:50.898083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 570/1176] 10.222.0.78 () {56 vars in 1183 bytes} [Wed Aug 24 09:48:50 2022] GET /identity/v3/domains/024daec882c543aca369fb983239b49d => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:50.904244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:50.917442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:51.022313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e75e8-9047-4415-9028-92640bf7ec73 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:51.064627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:51.065233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:51.065509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:51.065785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:51.066450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:51.067955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123282c-f5a7-419f-9d10-0f90f7e1dcaa None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:51.091856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 607/1177] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:48:50 2022] POST /identity/v3/users => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:48:53.123673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53135c68-5a0b-472b-8a5a-da2d71dbc5b2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:53.124278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53135c68-5a0b-472b-8a5a-da2d71dbc5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:53.124278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53135c68-5a0b-472b-8a5a-da2d71dbc5b2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:53.125654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 571/1178] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:53.131249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:53.131713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:53.131829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:53.150196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:53.151398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:53.213907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:53.223543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 608/1179] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:53.495055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53135c68-5a0b-472b-8a5a-da2d71dbc5b2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:53.608183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53135c68-5a0b-472b-8a5a-da2d71dbc5b2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:53.643507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:53.644173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:53.644456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:53.644731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:53.655237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:53.660686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:53.661921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:53.662976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 572/1180] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:53.668016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:53.773912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e2385-d2e6-4708-bce2-8396f387f9bb None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:53.809909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:53.810673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:53.810948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:53.811317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:53.812166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:53.812496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:53.815198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:53.824238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 609/1181] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:53.830779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:53.854556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:53.935216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e56c2-3749-4dab-b13e-684a6398fcc4 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:53.971160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:53.971827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:53.972092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:53.972361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] PATH_INFO: `/v3/users/3c9859f93a014de9bb62f4d27c2566b6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:53.985914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9859f93a014de9bb62f4d27c2566b6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:53.987390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:53.989093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 573/1182] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:53 2022] GET /identity/v3/users/3c9859f93a014de9bb62f4d27c2566b6 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:53.995070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:54.062009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:54.108087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dbddd-0c46-403b-9bb3-b5e630e0fea3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:54.144742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:54.145445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:54.145612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:54.145876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:54.221380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:54.222888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:54.287611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:54.333330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 610/1183] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:48:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:48:54.339329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:54.423451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:54.446697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55677f-409b-47a0-8e44-509a79f7a608 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:54.484158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:54.484841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:54.485247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:54.485521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] PATH_INFO: `/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:54.486653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] RBAC: Authorizing `identity:get_project(project_id=f50ec959b0c04b59af8a97e5b3dc99ba)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:54.491060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d2479-8f2d-4f55-a84a-b4d6c6da6a29 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:54.491060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 574/1184] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:54 2022] GET /identity/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:54.496120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:54.601626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff5c3-0ec1-44cf-9c51-2f0d2767b6bf None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:54.636342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:54.636950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:54.637234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:54.637506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:54.638230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=f50ec959b0c04b59af8a97e5b3dc99ba, user.id=3c9859f93a014de9bb62f4d27c2566b6) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:54.638535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:54.639988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39d4e5-4434-452a-9889-264e17da1a21 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:54.646319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 611/1185] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:48:54 2022] GET /identity/v3/role_assignments?user.id=3c9859f93a014de9bb62f4d27c2566b6&scope.project.id=f50ec959b0c04b59af8a97e5b3dc99ba&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:56.624824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0971d7-8dbe-4804-981f-32ef620a63c7 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:56.625415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0971d7-8dbe-4804-981f-32ef620a63c7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:56.625415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0971d7-8dbe-4804-981f-32ef620a63c7 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:56.627207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 575/1186] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:56.633233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b98804-0414-4a36-849b-61554332afff None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:56.634003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b98804-0414-4a36-849b-61554332afff None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:56.634003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b98804-0414-4a36-849b-61554332afff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:56.655399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-19b98804-0414-4a36-849b-61554332afff None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:56.657904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-19b98804-0414-4a36-849b-61554332afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:56.679046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:56.737798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 612/1187] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:48:57.012761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0971d7-8dbe-4804-981f-32ef620a63c7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.128956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0971d7-8dbe-4804-981f-32ef620a63c7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.165106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.165732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.166014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.166280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:57.177342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:48:57.182011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:48:57.183137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:48:57.184241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 576/1188] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:48:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:48:57.189805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b98804-0414-4a36-849b-61554332afff None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.317737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b98804-0414-4a36-849b-61554332afff None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.361148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.361764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.362038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.362420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:57.363094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:48:57.363394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:57.364799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:57.373360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 613/1189] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:48:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:57.379302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.388622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:57.493495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab13579-78a9-4659-92bb-947c12960ed3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.530167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.530798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.531152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.532170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] PATH_INFO: `/v3/users/3c9859f93a014de9bb62f4d27c2566b6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:57.534157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9859f93a014de9bb62f4d27c2566b6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:57.535582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:57.537300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 577/1190] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:48:57 2022] GET /identity/v3/users/3c9859f93a014de9bb62f4d27c2566b6 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:57.543206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.560420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:57.655242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e39b1-8de2-411f-a727-14b82b8e5f15 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.691571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.692256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.692531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.692802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] PATH_INFO: `/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:57.693925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] RBAC: Authorizing `identity:get_project(project_id=f50ec959b0c04b59af8a97e5b3dc99ba)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:57.695369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66d28b-59d2-4e4e-b9eb-dc148191b297 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:57.697088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 614/1191] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:48:57 2022] GET /identity/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:48:57.704101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:48:57.792468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:48:57.819777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3aed-fbc7-48e5-9581-41c514694070 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:48:57.854685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:48:57.855355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:57.855661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:57.855970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] PATH_INFO: `/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba/users/3c9859f93a014de9bb62f4d27c2566b6/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:57.858051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] RBAC: Authorizing `identity:create_grant(project_id=f50ec959b0c04b59af8a97e5b3dc99ba, user_id=3c9859f93a014de9bb62f4d27c2566b6, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:48:57.860204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd73664-c07b-4fc3-a3e9-459271bfc656 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:48:57.873382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 578/1192] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:48:57 2022] PUT /identity/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba/users/3c9859f93a014de9bb62f4d27c2566b6/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:48:59.879301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ef32-8078-448a-b612-6edbfd10cc84 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:59.879832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ef32-8078-448a-b612-6edbfd10cc84 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:59.879832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0361ef32-8078-448a-b612-6edbfd10cc84 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:59.881143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 615/1193] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:48:59.886449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:48:59.886871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:48:59.887003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:48:59.905524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:48:59.906764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:48:59.972809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 579/1194] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:48:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:49:00.250314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361ef32-8078-448a-b612-6edbfd10cc84 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:00.336004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:00.355598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0361ef32-8078-448a-b612-6edbfd10cc84 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:00.390850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:00.391477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:00.391756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:00.392081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:00.402693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:49:00.407532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:49:00.408769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:49:00.410039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 616/1195] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:49:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:49:00.415761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:00.497965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:00.548298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac3ee3-443f-49a2-a211-d760dc943326 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:00.583768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:00.584406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:00.584685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:00.584955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:00.585627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:00.585930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:00.587302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:00.596956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 580/1196] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:49:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:00.604004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:00.727232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b4c12a-96ab-4922-85d9-8f71029dbc14 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:00.769605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:00.770243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:00.770524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:00.770799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] PATH_INFO: `/v3/users/3c9859f93a014de9bb62f4d27c2566b6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:00.771967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] RBAC: Authorizing `identity:get_user(user_id=3c9859f93a014de9bb62f4d27c2566b6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:00.773431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:00.775046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 617/1197] 10.222.0.78 () {56 vars in 1177 bytes} [Wed Aug 24 09:49:00 2022] GET /identity/v3/users/3c9859f93a014de9bb62f4d27c2566b6 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:00.781763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:00.895975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad06876-fa38-4a81-b3ba-152f660b726a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:00.924066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:00.934129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:00.934764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:00.935046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:00.935322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:01.007736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:01.009227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:01.122755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 581/1198] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:49:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:49:01.129025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:01.146251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:01.240534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c47445-9c53-4dab-8b89-7ace811fabf9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:01.277940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:01.278580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:01.278877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:01.279163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] PATH_INFO: `/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:01.280340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] RBAC: Authorizing `identity:get_project(project_id=f50ec959b0c04b59af8a97e5b3dc99ba)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:01.283004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb428b7-71d2-4972-aa51-d3df71ce7a8e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:01.285232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 618/1199] 10.222.0.78 () {56 vars in 1186 bytes} [Wed Aug 24 09:49:01 2022] GET /identity/v3/projects/f50ec959b0c04b59af8a97e5b3dc99ba => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:01.292610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:49:01.322330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:01.402109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835ef9-0319-43ef-be54-d985da04415e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:49:01.442356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:49:01.443051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:01.443342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:01.443618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:01.444463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=f50ec959b0c04b59af8a97e5b3dc99ba, user.id=3c9859f93a014de9bb62f4d27c2566b6) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:49:01.444773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:49:01.446227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed01102-5234-45ab-9fbb-9477214f8ac6 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:49:01.453689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 582/1200] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:49:01 2022] GET /identity/v3/role_assignments?user.id=3c9859f93a014de9bb62f4d27c2566b6&scope.project.id=f50ec959b0c04b59af8a97e5b3dc99ba&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:49:11.273502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=61 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:11.450082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=55 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:49:59.718501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104ee6-4e83-4eea-a2e5-29e6d11716a4 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.718501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104ee6-4e83-4eea-a2e5-29e6d11716a4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.718501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c104ee6-4e83-4eea-a2e5-29e6d11716a4 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.718501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 619/1201] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:49:59.726012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:49:59.726519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:49:59.726606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:49:59.748048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:49:59.750025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:49:59.838488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 583/1202] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:49:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:00.218458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c104ee6-4e83-4eea-a2e5-29e6d11716a4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.397455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c104ee6-4e83-4eea-a2e5-29e6d11716a4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.439477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.440187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.440525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.440734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.453571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.457674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:00.472573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:00.473634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:00.474653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:00.474653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:00.474653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:00.478877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 620/1203] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:00.480220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.617567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a8ea6-6905-4a61-84d5-29f3266fad7a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.656547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.663137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d379a-529f-4360-8470-feda950565e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.663737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d379a-529f-4360-8470-feda950565e5 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.664057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d379a-529f-4360-8470-feda950565e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.664328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d379a-529f-4360-8470-feda950565e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.664974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d379a-529f-4360-8470-feda950565e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:00.665274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d379a-529f-4360-8470-feda950565e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.666668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d379a-529f-4360-8470-feda950565e5 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.679593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 584/1204] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:00.688120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:00.750713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:00.809358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8adf6b-1301-4665-ba28-7eff2ae722f6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:00.852335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:00.853318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:00.853794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:00.854201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:00.854988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:00.856606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cbba1-4637-4f54-a3f4-77fa8a3992cd None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:00.885747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 621/1205] 10.222.0.78 () {60 vars in 1132 bytes} [Wed Aug 24 09:50:00 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:03.099726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7bb7-34dd-4b0e-95aa-1ee3e54c4572 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.100434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7bb7-34dd-4b0e-95aa-1ee3e54c4572 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.100434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d7bb7-34dd-4b0e-95aa-1ee3e54c4572 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.101924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 585/1206] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:03.108758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.109446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.109575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.136296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:03.137631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:03.224242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 622/1207] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:03.517313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d7bb7-34dd-4b0e-95aa-1ee3e54c4572 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.658022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d7bb7-34dd-4b0e-95aa-1ee3e54c4572 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.705380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.706119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.706375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.706600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.720793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:03.729533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:03.730543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:03.731609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 586/1208] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:03.739399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:03.792593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:03.873869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068c988-df61-4011-ae1e-04e6e193b889 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:03.916309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75979315-f927-471c-8516-57073c474be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:03.918217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979315-f927-471c-8516-57073c474be7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:03.918217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979315-f927-471c-8516-57073c474be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:03.918217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75979315-f927-471c-8516-57073c474be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:03.918470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75979315-f927-471c-8516-57073c474be7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:03.918684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75979315-f927-471c-8516-57073c474be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:03.920048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75979315-f927-471c-8516-57073c474be7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:03.931833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 623/1209] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:03.936566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.002746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.071098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04fab-a8f3-40e7-a688-86f15af9dab9 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.117817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.118865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.118865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.119057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.131994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.135620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:04.154153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:04.156409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:04.157329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:04.157329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:04.157329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:04.157329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 587/1210] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:04.162013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75979315-f927-471c-8516-57073c474be7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.293163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.297711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75979315-f927-471c-8516-57073c474be7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.341502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.342912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.343195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.343536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.344310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:04.344618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.346324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.362815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 624/1211] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:04.368925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.503211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b416-26bd-4316-8ddb-7754a75e7953 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.504935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:04.548373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.549024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.549295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.549682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.558576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.560056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:04.575448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:04.576737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:04.578126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 588/1212] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:04.583722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.716780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfea794-88c8-49f1-8041-0dd42468a41c None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.756256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.756932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.757179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.757483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:04.758351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:04.758660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:04.760138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:04.786133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 625/1213] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:04.787523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:04.922155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88947bd-8bbb-46fb-bb91-fdbe5ddc043a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:04.965660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:04.966319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:04.966630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:04.967107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.006700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.058238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.060080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.187150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 589/1214] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:50:05.193154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.216792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.317270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99678d-30f4-49d5-ba83-0ff95ac8d407 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.359574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.361371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.361371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.361371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.372557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.374793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:05.393700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:05.395196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:05.396628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:05.396628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:05.396628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:05.400967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 626/1215] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:05.401330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.472546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.516618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5f56-1449-45c2-a340-32523fcc2fe8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.554111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.554734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.555017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.555292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.556391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:05.557056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.558639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.573078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 590/1216] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:05.579354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.673443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.710016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f1558-bbf0-44b0-bf72-8a05534997b9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.752375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.753170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.753447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.753720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.763466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.765426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:05.782990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:05.784171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:05.785973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 627/1217] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:05.790627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:05.934853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d562fb2-2c5f-4b8f-90b8-2ee320dedfba None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:05.963436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:05.978421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:05.979038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:05.979331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:05.979583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:05.980309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:05.980611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:05.982080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027d97e-a7ca-4c73-afcf-3faf49776a5d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:05.997327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 591/1218] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:06.005123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:06.125207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab919b6-3e22-4620-9a40-f102772b9e50 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:06.160253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:06.169174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:06.170704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:06.171157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:06.171621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:06.172954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b0a88b1bdce5483c95b59921dbfc5660) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:06.173467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:06.175678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c5c263-808c-46b2-882f-46ac1afceea6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:06.183130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 628/1219] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:06 2022] GET /identity/v3/role_assignments?user.id=b0a88b1bdce5483c95b59921dbfc5660&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:08.241371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce47c-5901-4bad-b44a-14077df7eb94 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.242013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce47c-5901-4bad-b44a-14077df7eb94 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.242013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce47c-5901-4bad-b44a-14077df7eb94 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.243415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 592/1220] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:08.249294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.249793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.250036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.268914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:08.270101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:08.340940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 629/1221] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:50:08.614042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce47c-5901-4bad-b44a-14077df7eb94 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:08.724915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce47c-5901-4bad-b44a-14077df7eb94 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:08.765564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:08.766241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.766483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.766778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.777278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:08.782284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:08.783438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:08.784364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:08.787956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 593/1222] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:08.789421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:08.898130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c068a0-a3b3-4ec6-a3ea-7d17e96c9926 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:08.924326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:08.935265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:08.936407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:08.936686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:08.936959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:08.937630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:08.937935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:08.939312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:08.948264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 630/1223] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:08.955190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.003278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.061499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75869a5-ecb0-4c64-a921-e4c5bb926bb3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.099645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.100524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.100845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.101085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.112762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.114349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:09.131405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:09.137210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:09.138377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:09.138377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:09.138377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:09.138377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 594/1224] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:09.138680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.250371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e8828-5f85-45a4-8cb5-27025da068d1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.290907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.291509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.291846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.292122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.292760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:09.293077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.294414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.306317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 631/1225] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:09.312274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.418191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.426760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe77637-7851-4194-bdd8-cbdfa2e02c48 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.469698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.470595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.470884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.471192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.479995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.481497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:09.495969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:09.497439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:09.499532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 595/1226] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:09.504254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.554733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:09.618680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf594-eacf-45ed-a366-5624dabbca45 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.655453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.656092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.656366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.656630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.657360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:09.657660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.659080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.677370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 632/1227] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:09.683093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.801700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302646-6531-43d0-8076-4ca7d9366efc None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.841271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:09.841931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:09.842363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:09.842681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:09.855760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:09.857333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:09.873145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:09.876643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:09.877764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:09.877764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:09.877764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:09.877764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 596/1228] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:09.881033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:09.990504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f367480-1952-491e-bfb9-22aa792699ff None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:09.993040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.027939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.028557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.028888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.029313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.030582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:10.030879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.032292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.044166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 633/1229] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.053146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.064454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.170895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc5210-8a4b-443a-b313-23613878ce6e None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.209384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.210100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.210372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.210904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.219525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.221539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:10.236869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:10.238237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:10.239319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 597/1230] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:10.244433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.356421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be942b7-0733-4c6e-a044-25fee6f0dcad None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.393575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.394267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.394478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.394748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.395445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:10.395768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.397220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bdeb5-0d14-425e-9539-f7f0759f32d6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.410551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 634/1231] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:10.416798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:10.493063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:10.532048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0961b6a8-3a04-45b3-9454-e7f607050277 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:10.571334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:10.572035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:10.572312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:10.572583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/b0a88b1bdce5483c95b59921dbfc5660/roles/5df8740d1dae485c831eb9d285979486` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:10.592103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=b0a88b1bdce5483c95b59921dbfc5660, role_id=5df8740d1dae485c831eb9d285979486)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:10.593591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc741b8e-97d7-4e59-be4f-ad5593ce6244 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:10.610894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 598/1232] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:10 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/b0a88b1bdce5483c95b59921dbfc5660/roles/5df8740d1dae485c831eb9d285979486 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:50:12.667117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613174-2304-493d-82b7-236723d82954 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.667647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613174-2304-493d-82b7-236723d82954 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.667647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21613174-2304-493d-82b7-236723d82954 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.669124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 635/1233] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:12.675294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:12.675752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:12.675870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:12.694095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:12.695305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:12.769647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 599/1234] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:13.055749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21613174-2304-493d-82b7-236723d82954 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.086504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.181136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21613174-2304-493d-82b7-236723d82954 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.217835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.218467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.218743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.219011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.229604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:13.236572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:13.237759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:13.238686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 636/1235] 10.222.0.78 () {56 vars in 1102 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:13.245792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.295149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.355645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718d816-5864-49e8-95a4-e3d1ae0dcf0e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.393475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.394214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.394580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.394925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.395697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:13.396076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.397556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.406721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 600/1236] 10.222.0.78 () {56 vars in 1103 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:13.412469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.512065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.532761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8ca5a-5242-4eae-862d-ddd4cca40f91 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.572951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.573580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.573854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.574124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.587613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.589334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:13.604027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:13.605100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:13.606124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:13.606124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:13.606124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:13.606124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 637/1237] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:13.611883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.740587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5f63a1-782f-478a-8436-f1e4d1a7ae54 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:13.751119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:13.785160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:13.785852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:13.786067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:13.786337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:13.786985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:13.787288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:13.788711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:13.802499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 601/1238] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:13.808637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:13.948783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423b102d-fb03-4909-b22e-5e8f425a54b1 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.001834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.003042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.003478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.003975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.009001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.017646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.019263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:14.033051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:14.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:14.035642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:14.039819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 638/1239] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:14.040941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.161422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa0c63-cbe3-4d89-a850-c4aad1f12f96 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.203073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.203615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.203939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.204217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.205219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:14.205524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.206964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.226186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 602/1240] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:14.232498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.349563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de835d5e-a84e-4330-b9d8-65496ee21760 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.389321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.389939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.390210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.390475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.463583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.471814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.473449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.604487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 639/1241] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 09:50:14.610587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.640776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.727413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6cbfce-6a25-41c1-9e72-425156b4556d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.769453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.770142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.770367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.770642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.783590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.785267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:14.802902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:14.804492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:14.805795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:14.805795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:14.805795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:14.805795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 603/1242] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:14.814174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:14.932819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:14.940568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f11f8-bd15-4a11-b5fc-08a070d5410d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:14.979559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:14.980232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:14.980670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:14.981110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:14.982007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:14.982315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:14.984074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.001565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 640/1243] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:15.002779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.087553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:15.115865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fbf11-faa8-44dd-8387-d25b45251ee8 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.164550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.165193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.165469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.165743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.173562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.175031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:15.188975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:15.190077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:15.191118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 604/1244] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:15.197001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.315941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50a218-c1e2-44fa-ab2d-85a8a6a080e9 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.354516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.355344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.355655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.356073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.356950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:15.357260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.359083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472a7e-1131-4d2d-8023-c25ed0609b26 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.374399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 641/1245] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:50:15.379494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:15.496957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9eed8-0ea5-4bbf-8670-3559f42e6275 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:15.534085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:15.534701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:15.534978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:15.535247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:15.536002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] RBAC: Adding query filter params (role.id=5df8740d1dae485c831eb9d285979486, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b0a88b1bdce5483c95b59921dbfc5660) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:15.536312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:15.537761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa6b22e-52b7-471f-a9ca-bbb5d34a351d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:15.544498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 605/1246] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:15 2022] GET /identity/v3/role_assignments?user.id=b0a88b1bdce5483c95b59921dbfc5660&scope.project.id=35e2575a816145b297430e41f586390d&role.id=5df8740d1dae485c831eb9d285979486 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:17.715632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e45444-3bf1-4788-8d58-bb772cc36cd3 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.716197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e45444-3bf1-4788-8d58-bb772cc36cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.716197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e45444-3bf1-4788-8d58-bb772cc36cd3 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.717811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 642/1247] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:17.724396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:17.724898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:17.724988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:17.731836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:17.746543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:17.747761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:17.827192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 606/1248] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:18.138440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e45444-3bf1-4788-8d58-bb772cc36cd3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.160438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:18.263449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e45444-3bf1-4788-8d58-bb772cc36cd3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.305138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.305822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.306098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.306367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.319040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:18.323709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:18.324854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:18.326015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 643/1249] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:18.331485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.456908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4f519-ed4b-4af9-a8f3-b1337b5bebdf None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.493942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:18.502233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.502937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.503218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.503509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.504223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:18.504531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:18.506084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:18.517229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 607/1250] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:18.523551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.617567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:18.644996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c25dc-616b-4a75-a2b6-c3c567f844e6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.684238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.685359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.688531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.688873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.700408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:18.702556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:18.718961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:18.720705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:18.722559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:18.722559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:18.722559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:18.722559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 644/1251] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:18.726906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:18.843639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d522c06-6a8c-4061-88e6-e5786d3c626d None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:18.889864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:18.890540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:18.890842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:18.891099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:18.891825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:18.892199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:18.893560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:18.906677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 608/1252] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:18.912285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.025136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb1182-5faa-4115-8a2d-27b400fddf81 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.054571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.063867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.064889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.065071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.065401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.074831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.076389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:19.088791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:19.089987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:19.091055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 645/1253] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:19.096458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.159831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.216047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f44ec-21c0-4205-84ae-252e81728597 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.259145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.259968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.260252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.260526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.261282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:19.261588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.263060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.285564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 609/1254] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:19.292713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.420961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c176d8-34e0-441f-9b87-918308f68729 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.470359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.471141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.471632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.472201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.541949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.557961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.559811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.682673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 646/1255] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:50:19.689673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:19.836180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222f1d8-4c37-42b7-b553-11aa3aa59a76 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:19.859341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:19.899089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:19.900129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:19.900573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:19.901008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:19.914470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:19.916050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:19.932460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:19.934313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:19.936066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:19.936066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:19.936066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:19.936066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 610/1256] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:19.940518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.078585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.095573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddcad1b-737c-4762-b64b-e1bac9927968 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.150300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.151335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.151917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.152358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.153725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.154194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.156985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.177203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 647/1257] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.183990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.368180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f782549-32d4-4cdf-a4d4-85071314e7d7 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.420672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.421422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.421602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.421874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.428900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.430336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:20.441648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:20.442803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:20.444569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 611/1258] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:20.449445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.563269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1fd57-3148-4fa4-85a5-e1fe8901c4d6 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.601966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.602808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.603130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.603414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.604175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.604561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.606613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90912dc1-41a2-4f62-af3a-a9c3f310702b None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.630830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 648/1259] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:20.637794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:20.696059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:20.819305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1cff9d-85b7-4ebb-9572-4fad2139f166 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:20.867641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:20.868577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:20.868781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:20.869182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:20.870105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b0a88b1bdce5483c95b59921dbfc5660) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:20.870496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:20.872384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b0ffad-d5ad-469f-bbd3-dac331911abc None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:20.881531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 612/1260] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:20 2022] GET /identity/v3/role_assignments?user.id=b0a88b1bdce5483c95b59921dbfc5660&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:22.918489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5929-5c6c-40f9-96c6-243847991ce2 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:22.919076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5929-5c6c-40f9-96c6-243847991ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:22.919076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5929-5c6c-40f9-96c6-243847991ce2 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:22.921422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 649/1261] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:22.929658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:22.930447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:22.930619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:22.951857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:22.953228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:23.041649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 613/1262] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:23.310337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5929-5c6c-40f9-96c6-243847991ce2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:23.322041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:23.422655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5929-5c6c-40f9-96c6-243847991ce2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:23.468347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:23.469234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:23.469510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:23.469779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:23.481898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:23.487303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:23.488573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:23.489696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 650/1263] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:23.494839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:23.514836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:23.604884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abc14c-f53d-4774-83ec-35b36bf90e09 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:23.642351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:23.642988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:23.643246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:23.643524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:23.644709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:23.644709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:23.645887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:23.654538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 614/1264] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:23.661154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:23.741274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:23.791237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2512adf0-1cb3-40f7-a87b-d1bdae7d3695 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:23.849440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:23.850455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:23.850754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:23.851385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:23.868578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:23.870329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:23.894115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:23.895478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:23.896847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:23.896847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:23.896847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:23.896847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 651/1265] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:23.902643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:23.999565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:24.020936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc38079-cee8-4440-b850-9100702eb175 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:24.056989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:24.057592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.057920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.058191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.059009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:24.059772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:24.060728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:24.073401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 615/1266] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:24.079525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:24.190962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a97af-a02e-46b6-be51-cde9b143d358 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:24.215587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:24.238930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:24.239575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.239878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.240151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.248933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:24.250882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:24.262872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:24.265326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:24.267184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 652/1267] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:24.271512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:24.397283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c23c-5100-40d5-b439-466037ae31ee None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:24.435415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:24.436062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.436374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.437143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.437997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:24.438488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:24.440166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:24.450897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:24.460716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 616/1268] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:24.466451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:24.573240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258daefc-7b2b-4627-aaf2-ef35f3ee6b22 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:24.610952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:24.611599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.611910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.612187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.627223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:24.629403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:24.645750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:24.646952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:24.647929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:24.647929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:24.647929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:24.647929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 653/1269] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:24.652774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:24.758916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085527cb-c303-4ed7-9c6e-d7cd82f89a3a None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:24.794762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:24.795367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:24.795850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:24.796234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:24.796882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:24.797186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:24.798558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:24.811998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 617/1270] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:24.818719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:24.868764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:24.965975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26198-7d56-4489-91f9-ec49f1ddc8bf None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.005299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.005934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.006214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.006486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.015910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.018446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:25.032908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:25.034413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:25.036203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 654/1271] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:25.041839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.109957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.162958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8cc2c-58c2-4a77-b0cd-bfecaca685a2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.217695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.218381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.218788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.219062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.219777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:25.220110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.221559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364da9c-ea1d-4656-ad13-94623c7719e3 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.234945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 618/1272] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:25.241074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:25.339041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:25.350474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff5604-4d3c-49d9-8c41-ed8b391d5023 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:25.395086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:25.395751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:25.396052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:25.396332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] PATH_INFO: `/v3/projects/35e2575a816145b297430e41f586390d/users/b0a88b1bdce5483c95b59921dbfc5660/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:25.398260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] RBAC: Authorizing `identity:create_grant(project_id=35e2575a816145b297430e41f586390d, user_id=b0a88b1bdce5483c95b59921dbfc5660, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:25.399784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f5d9f-cd2e-4db1-99b5-26a07f9e9b6e None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:25.413507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 655/1273] 10.222.0.78 () {58 vars in 1439 bytes} [Wed Aug 24 09:50:25 2022] PUT /identity/v3/projects/35e2575a816145b297430e41f586390d/users/b0a88b1bdce5483c95b59921dbfc5660/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:50:27.395259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ad754-3e40-473c-91cc-00899712636b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:27.395971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ad754-3e40-473c-91cc-00899712636b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:27.395971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ad754-3e40-473c-91cc-00899712636b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:27.397207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 619/1274] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:27.403775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:27.404392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:27.404392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:27.424675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:27.425904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:27.502240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 656/1275] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:27.776019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ad754-3e40-473c-91cc-00899712636b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:27.900611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ad754-3e40-473c-91cc-00899712636b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:27.921579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:27.940462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:27.941080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:27.941354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:27.941619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:27.953206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 09:50:27.958324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:27.960566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 09:50:27.962327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:27.963993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 620/1276] 10.222.0.78 () {56 vars in 1096 bytes} [Wed Aug 24 09:50:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:27.965533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.064860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:28.087752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952a6c65-b4a3-43d9-973a-2a65ba076c64 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:28.127210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:28.127914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.128210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.128476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.129153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:28.129612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:28.131732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:28.145034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 657/1277] 10.222.0.78 () {56 vars in 1099 bytes} [Wed Aug 24 09:50:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:28.151572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.275805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a5a59-0e90-45bc-a49e-cbd81729dcd3 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:28.316390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:28.316952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.317231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.317505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.329338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:28.331083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:28.347129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:28.348409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:28.349498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:28.349498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:28.349498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:28.349840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 621/1278] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:28.356087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.479383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84feec-1295-4704-8c81-2b8372df77d7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:28.516985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:28.517670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.517988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.518300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.518999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:28.519344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:28.520886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:28.534324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 658/1279] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:28.540364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.574436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:28.652157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc49b45-d0c0-4d65-9f0d-0b0b89408676 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:28.694646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:28.695289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.695568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.695876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.705125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:28.706832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 09:50:28.724371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:28.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:28.726719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 622/1280] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 09:50:28.731580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:28.735712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:28.859137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d61a4-182f-4d8a-ba10-25690d73abee None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:28.895002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:28.895617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:28.895928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:28.896206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:28.896941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:28.897270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:28.898693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:28.919520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 659/1281] 10.222.0.78 () {56 vars in 1143 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:28.926155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.048373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.064087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa1e60-509c-424c-929e-f2c7e02de369 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.105664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.106281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.106567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.106844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.187377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.189261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.310288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.323023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 623/1282] 10.222.0.78 () {58 vars in 1282 bytes} [Wed Aug 24 09:50:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 09:50:29.330023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.407608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.464641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564afb6-c56a-4742-8773-df6710f6f8b5 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.521142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.522079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.522516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.523013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.539543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.541738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 09:50:29.561084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.563217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 09:50:29.565602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 09:50:29.565602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 09:50:29.565602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:29.565602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 660/1283] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.570668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.713490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcbdd2e-8123-471e-bf57-c7cf0c89f918 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.776011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.776850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.777214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.777581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.779692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:29.779692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.781857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.795567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 624/1284] 10.222.0.78 () {56 vars in 1109 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:29.803028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:29.923628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5914-b74a-4236-a4d7-bed7fcd74b9d None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:29.938379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:29.965044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:29.965565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:29.965845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:29.966110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:29.974095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:29.975541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 09:50:29.988389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:29.989508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: ERROR keystone.server.flask.application Aug 24 09:50:29.990728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 661/1285] 10.222.0.78 () {56 vars in 1111 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:29.995442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.056420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:30.107246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba9e3a-c446-44c0-bbd9-a829a8b51e81 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.142114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.142723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.143004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.143271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.143986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.144290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.145717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a886c19-1e18-4994-be56-7f553b9e57b7 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.158439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 625/1286] 10.222.0.78 () {56 vars in 1148 bytes} [Wed Aug 24 09:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:30.164777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:30.274849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b99a17-aa65-4691-ba4c-bfc1ae8984b7 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:30.310726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:30.311985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:30.312300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:30.312570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:30.313308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] RBAC: Adding query filter params (role.id=d58e514376d0402cb9e57a1f44d5d41d, scope.project.id=35e2575a816145b297430e41f586390d, user.id=b0a88b1bdce5483c95b59921dbfc5660) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:30.313606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:30.315163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8141200-92d6-4e4f-988c-e1f77a31bc9c None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:30.321395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 662/1287] 10.222.0.78 () {56 vars in 1374 bytes} [Wed Aug 24 09:50:30 2022] GET /identity/v3/role_assignments?user.id=b0a88b1bdce5483c95b59921dbfc5660&scope.project.id=35e2575a816145b297430e41f586390d&role.id=d58e514376d0402cb9e57a1f44d5d41d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:32.291594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea17-97b2-4f11-a1d5-49d9d6fd20c7 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.292307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea17-97b2-4f11-a1d5-49d9d6fd20c7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.292307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42ea17-97b2-4f11-a1d5-49d9d6fd20c7 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.293534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 626/1288] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:32.300645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.301203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.301447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.320678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:32.321880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:32.355672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.402764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 663/1289] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:32.688047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ea17-97b2-4f11-a1d5-49d9d6fd20c7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.797511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42ea17-97b2-4f11-a1d5-49d9d6fd20c7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:32.808575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:32.834102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:32.834785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:32.835127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:32.835373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] PATH_INFO: `/v3/services/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:32.836007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:32.837361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:32.847626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:32.849139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:32.850362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 627/1290] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:32.854930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:32.971757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37f1f-36ca-422e-8cd0-414de4e48ed7 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.015458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.016405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.016794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.017357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.017920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.018127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.019569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42685000-d0c4-44a6-be31-ebce8f64f4d8 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.025211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 664/1291] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 24 09:50:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:33.032377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:33.147805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbb2729-a6b3-4f64-8bd8-5c6b833d63ee None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:33.186150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:33.187428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:33.187701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:33.187999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] PATH_INFO: `/v3/services` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:33.188638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:33.188936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:33.190315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb4084-d287-473b-913e-89f81d27c940 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:33.195381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 628/1292] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 24 09:50:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:35.161457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8032e0-a8f5-4ba8-bbf8-3a4dc5f5c13f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.162014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8032e0-a8f5-4ba8-bbf8-3a4dc5f5c13f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.162014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8032e0-a8f5-4ba8-bbf8-3a4dc5f5c13f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.163364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 665/1293] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:50:35.169044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4406d-c1ce-4457-93f0-4b0b08a9c0d0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.169818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4406d-c1ce-4457-93f0-4b0b08a9c0d0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.169974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4406d-c1ce-4457-93f0-4b0b08a9c0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.189462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-22d4406d-c1ce-4457-93f0-4b0b08a9c0d0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:35.190668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-22d4406d-c1ce-4457-93f0-4b0b08a9c0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:35.223809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:35.270303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 629/1294] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 09:50:35.557144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8032e0-a8f5-4ba8-bbf8-3a4dc5f5c13f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:35.593724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:35.672228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8032e0-a8f5-4ba8-bbf8-3a4dc5f5c13f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:35.707804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:35.708409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:35.708686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:35.708952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:35.709598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:35.711012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f6046a-75d7-47d4-82f2-d297be816a57 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:35.726716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 666/1295] 10.222.0.78 () {60 vars in 1141 bytes} [Wed Aug 24 09:50:35 2022] POST /identity/v3/services => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 09:50:37.663148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32489cbb-1ec6-450b-b46d-57a301d7906b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.663732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32489cbb-1ec6-450b-b46d-57a301d7906b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.663732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32489cbb-1ec6-450b-b46d-57a301d7906b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.665079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 630/1296] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:37.670369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:37.670781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:37.671011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:37.689806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:37.691019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:37.766465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 667/1297] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:50:38.041101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32489cbb-1ec6-450b-b46d-57a301d7906b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.151903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32489cbb-1ec6-450b-b46d-57a301d7906b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.196638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.197266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.197538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.197805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.198383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.199757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:38.209784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:38.211027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:38.212338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 631/1298] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:38.216905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.235904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.317914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c49a8d-05c1-4ad7-b1f1-0a41537c117c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.353398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.354036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.354315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.354592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.355221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.355525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.357093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.362868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 668/1299] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.369018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.403264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.505349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f308da-67dd-4dce-9ac3-fa498adbbfc2 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.540707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.541315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.541595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.541870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.542575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] RBAC: Adding query filter params (interface=public, service_id=551321ad7b4144b689089335203410b5, region_id=RegionOne) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:38.542883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.544290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b462e-fc59-4973-9b96-087cec430b2d None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.549437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 632/1300] 10.222.0.78 () {56 vars in 1251 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/endpoints?service_id=551321ad7b4144b689089335203410b5&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:38.555106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:38.640949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:38.659957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09358dea-7f5d-4f78-9c10-1dcf7005a2f3 None admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:38.693774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:38.694436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:38.694636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:38.694904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:38.695508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:38.696907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49ffc47-663c-4e95-90f6-bb5b4f630699 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:38.707526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 669/1301] 10.222.0.78 () {56 vars in 1087 bytes} [Wed Aug 24 09:50:38 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:50:40.680681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70ae0-978f-4821-9448-2642deb9a6ee None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.681251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70ae0-978f-4821-9448-2642deb9a6ee None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.681463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a70ae0-978f-4821-9448-2642deb9a6ee None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.683355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 633/1302] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:50:40.690816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:40.691419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:40.691699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:40.711996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:50:40.713161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:50:40.792424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 670/1303] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:50:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 09:50:41.097569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a70ae0-978f-4821-9448-2642deb9a6ee None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.214855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.228702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a70ae0-978f-4821-9448-2642deb9a6ee None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.267626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.268363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.268642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.268913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.269494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.270864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 09:50:41.279816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 09:50:41.281401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application with Lock( Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self._enter() Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application return self.creator() Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application created_value = creator( Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: ERROR keystone.server.flask.application Aug 24 09:50:41.282604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 634/1304] 10.222.0.78 () {56 vars in 1117 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 09:50:41.289373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.392995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:41.420628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7796b-587d-4e90-898d-4d586b0e1860 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.461745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.462374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.462654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.462929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] PATH_INFO: `/v3/services` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.463577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:50:41.463910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.465303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a96612-93fd-478e-a042-5756e10c9f07 None admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.471245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 671/1305] 10.222.0.78 () {56 vars in 1116 bytes} [Wed Aug 24 09:50:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:50:41.477268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:50:41.614981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac077383-c6d3-4118-acf1-57f9fa357f31 None admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:50:41.667073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:50:41.668007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:50:41.668493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:50:41.668959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:50:41.669772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:50:41.672218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf47890-d325-4b90-9c56-ad9d88e1d1cc None admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:50:41.704441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 635/1306] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:50:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:50:51.468161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=30 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:50:51.696539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:49.959693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187727-dd04-4203-b5ba-fc389186474c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:49.961538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187727-dd04-4203-b5ba-fc389186474c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:49.962727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187727-dd04-4203-b5ba-fc389186474c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:49.966872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 672/1307] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:51:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:49.972237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868e52-2734-47c4-a7f7-ee75c179a785 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:49.973859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868e52-2734-47c4-a7f7-ee75c179a785 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:49.974292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52868e52-2734-47c4-a7f7-ee75c179a785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:50.140616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-52868e52-2734-47c4-a7f7-ee75c179a785 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:50.142141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-52868e52-2734-47c4-a7f7-ee75c179a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:50.267193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 636/1308] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:51:50.621913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:50.622388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:50.622825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:50.625856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 673/1309] 10.222.0.78 () {54 vars in 965 bytes} [Wed Aug 24 09:51:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:50.632931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc01e-9b2d-4987-b64b-bc8abb8ac736 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:50.633226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc01e-9b2d-4987-b64b-bc8abb8ac736 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:50.633581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770bc01e-9b2d-4987-b64b-bc8abb8ac736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:50.835251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-770bc01e-9b2d-4987-b64b-bc8abb8ac736 None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:50.836606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-770bc01e-9b2d-4987-b64b-bc8abb8ac736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:50.886381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:51.001131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 637/1310] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Aug 24 09:51:50 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:51:51.001971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:51.003832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:51.167248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb60e6-4ba1-4718-baab-e912e1ca575c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:51.256196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:51.256944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:51.257308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:51.257701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:51.259626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:51.326285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:51.430581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:51.432300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4940a5-d66c-4c5e-878e-4157d126c65b service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:51.470437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 674/1311] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:51:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:51:53.855241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:53.855837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:53.855837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:53.857196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 638/1312] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:53.862745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf26377-9e36-4140-a40b-fcb5135cfaf8 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:53.863161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf26377-9e36-4140-a40b-fcb5135cfaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:53.863412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf26377-9e36-4140-a40b-fcb5135cfaf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:53.954437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3cf26377-9e36-4140-a40b-fcb5135cfaf8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:53.955861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3cf26377-9e36-4140-a40b-fcb5135cfaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:54.062727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 675/1313] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:51:54.353425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:54.354978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:54.465975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac530-9c84-4f57-94c8-16524d2d5b25 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:54.543617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:54.546525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:54.547213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:54.547696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:54.548235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:54.549954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:54.669077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:54.670554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba30e40-218a-4fbe-a0ec-014038101f11 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:54.690787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 639/1314] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:51:56.950526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563999d-b487-42a5-8f59-0bc2fa7427e6 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:56.951116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563999d-b487-42a5-8f59-0bc2fa7427e6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:56.951116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563999d-b487-42a5-8f59-0bc2fa7427e6 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:56.952467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 676/1315] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:51:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:56.957843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339995f4-cca6-4d42-9097-da277e78f416 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:56.958307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339995f4-cca6-4d42-9097-da277e78f416 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:56.958414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339995f4-cca6-4d42-9097-da277e78f416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.028883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-339995f4-cca6-4d42-9097-da277e78f416 None None] MFA Rules not processed for user `389b0d3b207d47b1af90de57d916552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:57.030142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-339995f4-cca6-4d42-9097-da277e78f416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:57.083668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:57.129144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 640/1316] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:51:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:51:57.435710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:57.436029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:57.436306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.437679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 677/1317] 10.222.0.78 () {54 vars in 965 bytes} [Wed Aug 24 09:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:51:57.443729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba318c-2d67-4ae2-a09b-2cb759749824 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:57.444042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba318c-2d67-4ae2-a09b-2cb759749824 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:57.444323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba318c-2d67-4ae2-a09b-2cb759749824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.523006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-95ba318c-2d67-4ae2-a09b-2cb759749824 None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:51:57.524243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-95ba318c-2d67-4ae2-a09b-2cb759749824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:57.601535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 641/1318] 10.222.0.78 () {58 vars in 1062 bytes} [Wed Aug 24 09:51:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:51:57.610757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:51:57.612351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:57.622801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:57.721236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fea0cf-2b79-4148-a7a8-5d704c8fcbde None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:51:57.797652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:51:57.798254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:51:57.798529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:51:57.798794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:51:57.800515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:51:57.871415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:51:57.921653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:51:57.923092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0756b5d0-a7f7-4134-8c8e-b3d3235255ba service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:51:57.942331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 678/1319] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:51:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:00.825913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:00.826490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:00.826490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:00.827811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 642/1320] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:00.833301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df38b-21d1-4768-ad93-2cab509a3cb0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:00.834291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df38b-21d1-4768-ad93-2cab509a3cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:00.834291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580df38b-21d1-4768-ad93-2cab509a3cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:00.905775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-580df38b-21d1-4768-ad93-2cab509a3cb0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:00.907015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-580df38b-21d1-4768-ad93-2cab509a3cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:01.010073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 679/1321] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:52:01.297389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:01.298934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:01.350272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:01.404329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424de758-4830-46f5-9f7b-56100c4d4f5c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:01.492598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:01.493213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:01.493494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:01.493766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:01.495410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:01.601131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:01.617433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:01.618889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ba3c60-522a-4717-b2e1-f75628159ba4 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:01.637673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 643/1322] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:04.290391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:04.290932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:04.291026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:04.292494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 680/1323] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:04.298363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ace6f-d4c6-4f91-929d-bdff0d6888bc None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:04.298841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ace6f-d4c6-4f91-929d-bdff0d6888bc None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:04.298841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ace6f-d4c6-4f91-929d-bdff0d6888bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:04.377972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b06ace6f-d4c6-4f91-929d-bdff0d6888bc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:04.379210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b06ace6f-d4c6-4f91-929d-bdff0d6888bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:04.490302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 644/1324] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:04.776856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:04.778563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:04.799106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:04.891301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75474db-955b-4857-ad36-c0159c55c1b5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:04.971101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:04.971710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:04.972022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:04.972298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:04.973984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:05.053001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:05.095237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:05.096975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284b87d-abc3-431c-ac2c-ff71e2731364 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:05.117415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 681/1325] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:07.129373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:07.130139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:07.130139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:07.131182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 645/1326] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:07.138019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62d163-ef27-455f-9900-9aa859185b45 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:07.138533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62d163-ef27-455f-9900-9aa859185b45 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:07.139007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62d163-ef27-455f-9900-9aa859185b45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:07.218861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9e62d163-ef27-455f-9900-9aa859185b45 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:07.220141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9e62d163-ef27-455f-9900-9aa859185b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:07.330349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 682/1327] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:52:07.619767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:07.621351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:07.685933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:07.732351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c8ef4-b597-4c1f-b7cd-ec01e7bc9660 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:07.814147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:07.815034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:07.815311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:07.815577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:07.817256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:07.935770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:07.938782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:07.940618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db85e3a8-2322-4113-a828-7adcaa02b695 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:07.961953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 646/1328] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:09.968203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:09.968794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:09.968794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:09.970292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 683/1329] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:09.976486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1739-bedf-487c-91d2-aab1b2d24e67 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:09.977032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1739-bedf-487c-91d2-aab1b2d24e67 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:09.977032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1739-bedf-487c-91d2-aab1b2d24e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:10.051572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-456c1739-bedf-487c-91d2-aab1b2d24e67 None None] MFA Rules not processed for user `389b0d3b207d47b1af90de57d916552e`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:10.054007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-456c1739-bedf-487c-91d2-aab1b2d24e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:10.158939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 647/1330] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:09 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:10.454892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:10.456802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:10.496826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:10.581352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733b2cfe-3470-4f49-a7c7-e45b47cbff19 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:10.667878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:10.668487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:10.668760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:10.669026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:10.670619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:10.757409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:10.793494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:10.794869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a035ff0-a121-43f3-adfa-c5217b582644 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:10.815663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 684/1331] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:10 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:13.061541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:13.062117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:13.062117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:13.063524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 648/1332] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:52:13.070611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bca2e-798d-423b-a957-68d78c1be913 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:13.071138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bca2e-798d-423b-a957-68d78c1be913 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:13.071650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410bca2e-798d-423b-a957-68d78c1be913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:13.143682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-410bca2e-798d-423b-a957-68d78c1be913 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:13.144916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-410bca2e-798d-423b-a957-68d78c1be913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:13.250184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 685/1333] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:13.533663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:13.535210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:13.609365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:13.643509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81ed0d-9765-4262-ae07-4333af0afaeb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:13.726109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:13.727098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:13.727611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:13.728130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:13.731252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:13.843725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:13.845220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e19281-e688-46ee-bc27-e1925c31eb83 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:13.862291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:13.870974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 649/1334] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:52:15.916191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:15.916746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:15.916746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:15.918146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 686/1335] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:15.923509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04179-315b-4267-b5eb-0bd93c274138 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:15.923989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04179-315b-4267-b5eb-0bd93c274138 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:15.924110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a04179-315b-4267-b5eb-0bd93c274138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:16.004391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-02a04179-315b-4267-b5eb-0bd93c274138 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:16.006537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-02a04179-315b-4267-b5eb-0bd93c274138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:16.139861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 650/1336] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:16.428141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:16.430321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:16.492809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:16.559017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41f6fc-5a49-4c84-9d11-55528d393b87 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:16.638823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:16.639429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:16.639712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:16.640018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:16.641683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:16.732516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:16.760483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:16.761946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21d1dcc-c56b-4a98-8d37-05f17b987ed8 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:16.781335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 687/1337] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:19.120291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.120838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.120929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.122228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 651/1338] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:19.127820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a65-8f89-411a-a9fd-4e6884245474 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.128274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a65-8f89-411a-a9fd-4e6884245474 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.128391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f7a65-8f89-411a-a9fd-4e6884245474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.198746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e33f7a65-8f89-411a-a9fd-4e6884245474 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:19.200075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e33f7a65-8f89-411a-a9fd-4e6884245474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:19.302526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 688/1339] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:19.583158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:19.584784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:19.666176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:19.695568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e79cd69-2fcb-4afe-bc01-d2f7d9b6bed1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:19.770732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:19.771236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:19.771512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:19.771856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:19.774387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:19.892732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:19.894178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3069ab-cc9f-4116-b143-3e7eecda756d service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:19.915030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 652/1340] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:24.167658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:24.168375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:24.168375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:24.169659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 689/1341] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:52:24.176354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61bf-654e-49cf-8f3c-f55010e7dfd3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:24.176956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61bf-654e-49cf-8f3c-f55010e7dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:24.177310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e61bf-654e-49cf-8f3c-f55010e7dfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:24.187259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:24.256226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-414e61bf-654e-49cf-8f3c-f55010e7dfd3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:24.258189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-414e61bf-654e-49cf-8f3c-f55010e7dfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:24.373309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 653/1342] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:24.675030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:24.677596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:24.769950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:24.857506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c608d7-cd86-4809-bd0b-5621781fcde0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:24.952856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:24.953384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:24.953669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:24.953943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:24.955599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:25.088696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:25.117676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:25.120121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55932a0a-0d3c-4277-8b57-c744c9db7ace service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:25.144197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 690/1343] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:28.090233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.090785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.090785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.092087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 654/1344] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:28.098005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812cd89-1b2c-488d-92bb-d61c88c0f9e8 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.098537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812cd89-1b2c-488d-92bb-d61c88c0f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.098659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e812cd89-1b2c-488d-92bb-d61c88c0f9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.181986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e812cd89-1b2c-488d-92bb-d61c88c0f9e8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:28.183812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e812cd89-1b2c-488d-92bb-d61c88c0f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:28.350673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 691/1345] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:28.638070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:28.639645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:28.736248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:28.755853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef825691-9c53-4f0d-ade5-ffb0a4273593 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:28.836183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:28.837148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:28.837349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:28.837601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:28.839320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:28.967980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:28.969479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287475e-fa48-46d4-bcc5-8c8c391d0ae1 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:28.989182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 655/1346] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:33.438601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:33.439143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:33.439477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:33.441785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 692/1347] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:33.448939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f89ef-0cb6-4111-a416-8d900a5ae4b0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:33.449456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f89ef-0cb6-4111-a416-8d900a5ae4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:33.449929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f89ef-0cb6-4111-a416-8d900a5ae4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:33.471695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:33.547835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0d5f89ef-0cb6-4111-a416-8d900a5ae4b0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:33.547835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0d5f89ef-0cb6-4111-a416-8d900a5ae4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:33.695370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 656/1348] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:52:34.014240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:34.015808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:34.076507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:34.135412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df45d5-7ed0-488a-8dba-187502b6549c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:34.244996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:34.245974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:34.246759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:34.247220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:34.250347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:34.404614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:34.407054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:34.408936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6804429-af4f-4f90-838a-6ff49661aafe service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:34.429385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 693/1349] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:39.996849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048b904-fb3d-4ea5-b282-14b47ee04c2a None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:39.997791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048b904-fb3d-4ea5-b282-14b47ee04c2a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:39.998291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048b904-fb3d-4ea5-b282-14b47ee04c2a None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:39.999960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 657/1350] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 09:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:40.007279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac9d6-ccb0-4414-a151-379cd410e0fd None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:40.007771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac9d6-ccb0-4414-a151-379cd410e0fd None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:40.008331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ac9d6-ccb0-4414-a151-379cd410e0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:40.213595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e61ac9d6-ccb0-4414-a151-379cd410e0fd None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:40.215392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e61ac9d6-ccb0-4414-a151-379cd410e0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:40.366911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 694/1351] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 09:52:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:43.693889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=59 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:45.224622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c53cdf-050f-4976-90eb-b4ca55244d35 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:45.225397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c53cdf-050f-4976-90eb-b4ca55244d35 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:45.225497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c53cdf-050f-4976-90eb-b4ca55244d35 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:45.228623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 658/1352] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 09:52:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:45.236401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e60cf-83bc-4cfe-ab9d-26d50733de72 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:45.237496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e60cf-83bc-4cfe-ab9d-26d50733de72 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:45.237496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e60cf-83bc-4cfe-ab9d-26d50733de72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:45.331217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:45.360921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-506e60cf-83bc-4cfe-ab9d-26d50733de72 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:45.362155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-506e60cf-83bc-4cfe-ab9d-26d50733de72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:45.538867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 695/1353] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 09:52:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:46.638406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524d34-0b9d-4f98-bcba-38d40dd9dc24 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:46.639025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524d34-0b9d-4f98-bcba-38d40dd9dc24 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:46.639025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71524d34-0b9d-4f98-bcba-38d40dd9dc24 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:46.640342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 659/1354] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 09:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:46.645356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83034-42c0-4a73-95bb-2a3fe0f9f5d2 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:46.645755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83034-42c0-4a73-95bb-2a3fe0f9f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:46.645933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e83034-42c0-4a73-95bb-2a3fe0f9f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:46.757155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e4e83034-42c0-4a73-95bb-2a3fe0f9f5d2 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:46.758319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e4e83034-42c0-4a73-95bb-2a3fe0f9f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:46.809162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:46.883221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 696/1355] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 09:52:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:50.429464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdd47-e629-4328-918d-71a9b3c24cc2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:50.430034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdd47-e629-4328-918d-71a9b3c24cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:50.430034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bdd47-e629-4328-918d-71a9b3c24cc2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:50.431990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 660/1356] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:50.437769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db5b2-cb4c-4b79-8fcb-20e15e223372 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:50.438415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db5b2-cb4c-4b79-8fcb-20e15e223372 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:50.438415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2db5b2-cb4c-4b79-8fcb-20e15e223372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:50.522360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7b2db5b2-cb4c-4b79-8fcb-20e15e223372 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:50.523589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7b2db5b2-cb4c-4b79-8fcb-20e15e223372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:50.676508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 697/1357] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:51.059112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:51.059672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:51.060221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:51.062023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 661/1358] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 24 09:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:51.070264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23b08-2871-4e54-b54e-a656f3900574 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:51.070832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23b08-2871-4e54-b54e-a656f3900574 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:51.071731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae23b08-2871-4e54-b54e-a656f3900574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:51.097338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:51.170201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3ae23b08-2871-4e54-b54e-a656f3900574 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:51.171713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3ae23b08-2871-4e54-b54e-a656f3900574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:51.339892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 698/1359] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 24 09:52:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:52:51.348325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:51.349867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:51.526605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce038a-ba6e-4b55-b9d5-000a4bc2fcbe None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:51.637439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:51.729541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:51.730474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:51.730958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:51.731402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:51.734251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:51.894031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:51.897050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e39cc34-e7ee-4911-9ec2-96937888c17c service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:51.930781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 662/1360] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:52:54.298721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:54.299329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:54.299572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:54.302194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 699/1361] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:54.311194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf8e27-2778-4887-8d38-b8e831745eec None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:54.312531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf8e27-2778-4887-8d38-b8e831745eec None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:54.312997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebf8e27-2778-4887-8d38-b8e831745eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:54.404607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5ebf8e27-2778-4887-8d38-b8e831745eec None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:54.406110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5ebf8e27-2778-4887-8d38-b8e831745eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:54.464436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:54.559949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 663/1362] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:54.866839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:54.868496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:55.004998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:55.187503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210d70-d35d-4eed-9077-8850d874fa74 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:55.398032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:55.398642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:55.398929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:55.399206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:55.401603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:55.428546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:55.531860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:55.533819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff6d21d-48d0-4b23-b38f-33cdd78e3e4c service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:55.557917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 700/1363] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 691 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:57.946871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b6ae-c63c-4165-b8b4-c8f2ca6cfe9b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:57.947471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b6ae-c63c-4165-b8b4-c8f2ca6cfe9b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:57.947571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3b6ae-c63c-4165-b8b4-c8f2ca6cfe9b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:57.949236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 664/1364] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:52:57.954940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2bb85-1c46-4130-833d-4f9bfe1d1742 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:57.955442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2bb85-1c46-4130-833d-4f9bfe1d1742 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:57.955577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c2bb85-1c46-4130-833d-4f9bfe1d1742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:58.066602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-61c2bb85-1c46-4130-833d-4f9bfe1d1742 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:58.068603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-61c2bb85-1c46-4130-833d-4f9bfe1d1742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:58.235664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:58.239084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 701/1365] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:58.553148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:58.553592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:58.554002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:58.556598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 665/1366] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 24 09:52:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:58.567859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f77b6-cec2-4889-969d-8bd77dfc3904 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:58.568209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f77b6-cec2-4889-969d-8bd77dfc3904 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:58.568487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597f77b6-cec2-4889-969d-8bd77dfc3904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:58.668367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-597f77b6-cec2-4889-969d-8bd77dfc3904 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:58.669926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-597f77b6-cec2-4889-969d-8bd77dfc3904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:58.827654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 702/1367] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 24 09:52:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:52:58.835431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:52:58.837171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:59.051213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad961e2-5c10-42c6-b531-60829a1defc5 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:52:59.079500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:59.208328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:52:59.209253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:59.210000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:59.210427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:59.213275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:59.330075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:52:59.331930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d8e59-cb5d-4170-ae6d-383abb819526 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:52:59.355136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 666/1368] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:52:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:52:59.595956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5bad-55be-49d5-866c-86d8cc883865 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:59.596778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5bad-55be-49d5-866c-86d8cc883865 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:59.596778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5bad-55be-49d5-866c-86d8cc883865 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:59.598339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 703/1369] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:52:59.604673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3369-48b5-4e5d-a39c-b9ee73ad1e85 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:52:59.604939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3369-48b5-4e5d-a39c-b9ee73ad1e85 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:52:59.605292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c3369-48b5-4e5d-a39c-b9ee73ad1e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:52:59.641069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:52:59.683644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ce2c3369-48b5-4e5d-a39c-b9ee73ad1e85 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:52:59.684936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ce2c3369-48b5-4e5d-a39c-b9ee73ad1e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:52:59.799885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 667/1370] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:00.392233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:00.392718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:00.393194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:00.395624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 704/1371] 10.222.0.78 () {54 vars in 964 bytes} [Wed Aug 24 09:53:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:00.401912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212d8d-33d8-4557-a6fa-b7ba62fbbaed None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:00.402403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212d8d-33d8-4557-a6fa-b7ba62fbbaed None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:00.402613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54212d8d-33d8-4557-a6fa-b7ba62fbbaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:00.515113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-54212d8d-33d8-4557-a6fa-b7ba62fbbaed None None] MFA Rules not processed for user `6b171220975d48488be864348cc9aca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:00.516378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-54212d8d-33d8-4557-a6fa-b7ba62fbbaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:00.577393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:00.614725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 668/1372] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Aug 24 09:53:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:00.623782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:00.625523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:00.683264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:00.747895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a8990-1179-49db-8d85-25f027cb1df6 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:00.838773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:00.839526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:00.839900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:00.840378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:00.842469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:00.982270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:01.007748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:01.010228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6ece2-4ce3-4a24-b043-83da741c7e1d service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:01.042592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 705/1373] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:01.501845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:01.520854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:01.520854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:01.520854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 669/1374] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:01 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:01.521793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b9df-4f8e-4661-8066-261a1c9445eb None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:01.522165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b9df-4f8e-4661-8066-261a1c9445eb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:01.522641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d1b9df-4f8e-4661-8066-261a1c9445eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:01.605112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-50d1b9df-4f8e-4661-8066-261a1c9445eb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:01.606378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-50d1b9df-4f8e-4661-8066-261a1c9445eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:01.733376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 706/1375] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:02.029507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:02.031037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:02.199484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313e88-5fc7-407a-92ef-00cd2de39f6b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:02.255264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:02.331064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:02.331911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:02.332199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:02.332473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:02.334159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:02.434473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:02.436145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf368bc-ec9b-4f7d-8930-59f62fe313e3 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:02.458818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 670/1376] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:53:04.540985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8872b-f086-4814-a2f1-35394894787e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:04.542257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8872b-f086-4814-a2f1-35394894787e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:04.542257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c8872b-f086-4814-a2f1-35394894787e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:04.543917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 707/1377] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:04.549287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e46452-2487-4845-9a27-9890ef031f31 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:04.549822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e46452-2487-4845-9a27-9890ef031f31 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:04.549822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e46452-2487-4845-9a27-9890ef031f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:04.627817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-76e46452-2487-4845-9a27-9890ef031f31 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:04.629211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-76e46452-2487-4845-9a27-9890ef031f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:04.637532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:04.766687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 671/1378] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:05.074657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8872b-f086-4814-a2f1-35394894787e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:05.076241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-45c8872b-f086-4814-a2f1-35394894787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:05.149200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:05.250449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c8872b-f086-4814-a2f1-35394894787e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:05.372785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:05.373322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:05.373586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:05.373858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:05.375461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:05.396339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:05.485185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:05.486810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb2a25-35d4-4320-bbd7-549b350a9c45 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:05.507648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 708/1379] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:07.642831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:07.643417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:07.643417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:07.644934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 672/1380] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:07.650351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43cfbc-27a4-469c-b38a-cd3b2c074da6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:07.650823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43cfbc-27a4-469c-b38a-cd3b2c074da6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:07.650923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43cfbc-27a4-469c-b38a-cd3b2c074da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:07.733374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bf43cfbc-27a4-469c-b38a-cd3b2c074da6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:07.735590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bf43cfbc-27a4-469c-b38a-cd3b2c074da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:07.861722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:07.871334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 709/1381] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:08.178437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:08.180016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:08.380261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:08.398732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20e1f9-3db8-4ef8-b9a6-797f0197780e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:08.593111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:08.593815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:08.594166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:08.594503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:08.596216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:08.702141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:08.703928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21291a9b-b085-496e-8d79-257f3844583f service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:08.721316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:08.739229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 673/1382] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:11.084930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:11.085745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:11.085745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:11.086997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 710/1383] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:11.093702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb591ef2-2d2d-410f-bbe1-2cf2bd2dfabe None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:11.094364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb591ef2-2d2d-410f-bbe1-2cf2bd2dfabe None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:11.094364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb591ef2-2d2d-410f-bbe1-2cf2bd2dfabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:11.173543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-cb591ef2-2d2d-410f-bbe1-2cf2bd2dfabe None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:11.174819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cb591ef2-2d2d-410f-bbe1-2cf2bd2dfabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:11.296779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 674/1384] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:11.600689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:11.602123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:11.792657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acd7c9-0bf8-44d8-bc09-44abd7127bf8 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:11.901566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:11.952903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:11.953508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:11.953787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:11.954060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:11.955667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:12.078016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:12.081764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe421b0-d0bf-45ac-ba58-f543644c6bfc service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:12.112814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 711/1385] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:14.321245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:14.322676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:14.322676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:14.323589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 675/1386] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:53:14.329091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d408-e382-44a7-a5b6-cdfbbc809a6e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:14.329975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d408-e382-44a7-a5b6-cdfbbc809a6e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:14.329975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079d408-e382-44a7-a5b6-cdfbbc809a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:14.414698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a079d408-e382-44a7-a5b6-cdfbbc809a6e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:14.416738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a079d408-e382-44a7-a5b6-cdfbbc809a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:14.457378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:14.532417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 712/1387] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:53:14.846321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:14.848288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:14.940433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:15.019484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877153b-8813-45df-bfe5-94137eb8d9d1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:15.142168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:15.142780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:15.143059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:15.143326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:15.144970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:15.185758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:15.240309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:15.242219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627bbcc7-09ae-4713-afed-1a04b75214e6 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:15.262623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 676/1388] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:16.706939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238934e-8c5e-472a-bb97-1245359caf51 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:16.707494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238934e-8c5e-472a-bb97-1245359caf51 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:16.707494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238934e-8c5e-472a-bb97-1245359caf51 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:16.708765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 713/1389] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:16.715812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3008b4-daa8-4a20-9deb-acd36892f865 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:16.716312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3008b4-daa8-4a20-9deb-acd36892f865 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:16.716433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3008b4-daa8-4a20-9deb-acd36892f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:16.795386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-fc3008b4-daa8-4a20-9deb-acd36892f865 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:16.796681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fc3008b4-daa8-4a20-9deb-acd36892f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:16.906209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 677/1390] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:17.215474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc952fa-f75e-4cd4-9553-6cdf5601578f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:17.215899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc952fa-f75e-4cd4-9553-6cdf5601578f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:17.216242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc952fa-f75e-4cd4-9553-6cdf5601578f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:17.217703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 714/1391] 10.222.0.78 () {54 vars in 967 bytes} [Wed Aug 24 09:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:17.223420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:17.223705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:17.224008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:17.240789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:17.251821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad2e9-1cc2-4313-85ce-c35392849011 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:17.252109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad2e9-1cc2-4313-85ce-c35392849011 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:17.252387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ad2e9-1cc2-4313-85ce-c35392849011 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:17.254001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 715/1392] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:17.258922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15dc85-543e-4699-ba54-715543998462 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:17.259880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15dc85-543e-4699-ba54-715543998462 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:17.260298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15dc85-543e-4699-ba54-715543998462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:17.319932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] MFA Rules not processed for user `3061244943c84217bf9ba30224cd4951`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:17.321451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:17.363655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8d15dc85-543e-4699-ba54-715543998462 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:17.365352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8d15dc85-543e-4699-ba54-715543998462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:17.415692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 678/1393] 10.222.0.78 () {58 vars in 1064 bytes} [Wed Aug 24 09:53:17 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:17.423919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:17.433032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7cf6ee-3114-4b33-8a39-96b237daa924 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:17.494712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 716/1394] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:53:17.537398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:17.538398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:17.538912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:17.539415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:17.541863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:17.577584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:17.680420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:17.682630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4a25-2bd5-4412-bceb-ec3a1da4ccb3 service glance] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:17.703064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 679/1395] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:17.791913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15dc85-543e-4699-ba54-715543998462 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:17.793199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8d15dc85-543e-4699-ba54-715543998462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:17.808039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:17.940380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15dc85-543e-4699-ba54-715543998462 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:18.099876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:18.100503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:18.100783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:18.101049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:18.102710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:18.115489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:18.215488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:18.218123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d9cac8-7e43-4cba-88fd-7bff115c6e50 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:18.238826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 717/1396] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:18.489397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d73a4-4c2e-4ab6-9620-26cfed39400d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:18.489946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d73a4-4c2e-4ab6-9620-26cfed39400d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:18.489946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d73a4-4c2e-4ab6-9620-26cfed39400d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:18.569281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-551d73a4-4c2e-4ab6-9620-26cfed39400d None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:18.570527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-551d73a4-4c2e-4ab6-9620-26cfed39400d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:18.660720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:18.671539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 680/1397] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 09:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:18.718801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:18.719178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:18.719645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:18.721392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 718/1398] 10.222.0.78 () {54 vars in 943 bytes} [Wed Aug 24 09:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:18.728683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2074712-4346-479d-9432-ae2ae50b4159 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:18.729158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2074712-4346-479d-9432-ae2ae50b4159 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:18.729459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2074712-4346-479d-9432-ae2ae50b4159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:18.816792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f2074712-4346-479d-9432-ae2ae50b4159 None None] MFA Rules not processed for user `61f9de1f0b754d74bf44b92eb2134081`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:18.818033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f2074712-4346-479d-9432-ae2ae50b4159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:18.946995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 681/1399] 10.222.0.78 () {58 vars in 1040 bytes} [Wed Aug 24 09:53:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:18.957329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:18.958870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:19.130965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa3b18a-4e90-4c99-80da-77c3f36f4e53 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:19.133912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:19.258771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:19.259982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:19.260265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:19.260538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:19.262243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:19.357440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:19.359088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccc6cb-3a6c-447e-ac3d-54ba16ebc07a service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:19.371121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 719/1400] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 09:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 09:53:20.315454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:20.316053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:20.316053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:20.318785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 682/1401] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:20.327535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea652d9-a964-4644-9d47-eeda714c5dfe None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:20.328120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea652d9-a964-4644-9d47-eeda714c5dfe None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:20.328358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea652d9-a964-4644-9d47-eeda714c5dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:20.378719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:20.412097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1ea652d9-a964-4644-9d47-eeda714c5dfe None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:20.413549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1ea652d9-a964-4644-9d47-eeda714c5dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:20.549600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 720/1402] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:20.971643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:20.973751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:21.046804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:21.168362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a9a596-0241-40d8-8926-978a6e8eff35 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:21.325957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:21.326672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:21.326858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:21.327132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:21.328841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:21.352285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:21.443699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:21.450219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917bf4a-286a-4db9-b400-78741bce4ff0 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:21.467366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 683/1403] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:53:23.576636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:23.577274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:23.577676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:23.580243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 721/1404] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:23.587157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6141b6-bd98-4423-9cbb-a1167ac57a7a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:23.588282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6141b6-bd98-4423-9cbb-a1167ac57a7a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:23.588643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6141b6-bd98-4423-9cbb-a1167ac57a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:23.702501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-8f6141b6-bd98-4423-9cbb-a1167ac57a7a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:23.703965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8f6141b6-bd98-4423-9cbb-a1167ac57a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:23.830674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:23.855821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 684/1405] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:53:24.158761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:24.161194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:24.295601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:24.358189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97edbfe-f19d-44d5-bc41-068497d98745 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:24.525851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:24.527050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:24.527501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:24.527768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:24.529492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:24.628052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:24.650498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:24.652261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c8e25-6fbc-493c-9a21-36591baf227e service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:24.684441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 722/1406] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:26.754533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:26.754533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:26.754533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:26.754533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 685/1407] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:26.762448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36695-04ef-4fbe-8cc7-419b87d792cf None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:26.763005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36695-04ef-4fbe-8cc7-419b87d792cf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:26.763341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b36695-04ef-4fbe-8cc7-419b87d792cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:26.852504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-99b36695-04ef-4fbe-8cc7-419b87d792cf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:26.854151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-99b36695-04ef-4fbe-8cc7-419b87d792cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:27.005665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 723/1408] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:27.310621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:27.312713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:27.509487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc611e5-a982-4d06-a565-81f6b05c1859 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:27.636965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:27.701027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:27.701951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:27.702404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:27.703083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:27.704974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:27.808429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:27.810451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff58574-5dc1-4024-a88f-7ced05dfee57 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:27.830643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 686/1409] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:37.006123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=78 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:37.828162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=65 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:42.792444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e167e1-469c-4574-9418-c4a6af6bcdfe None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.793272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e167e1-469c-4574-9418-c4a6af6bcdfe None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.793272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e167e1-469c-4574-9418-c4a6af6bcdfe None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.794302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 724/1410] 10.222.0.78 () {54 vars in 918 bytes} [Wed Aug 24 09:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:53:42.802284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11fc-37b4-417f-88d6-3cfd730c02d1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:42.802876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11fc-37b4-417f-88d6-3cfd730c02d1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:42.803027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f11fc-37b4-417f-88d6-3cfd730c02d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:42.895075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a79f11fc-37b4-417f-88d6-3cfd730c02d1 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:42.897442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a79f11fc-37b4-417f-88d6-3cfd730c02d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:43.027832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 687/1411] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Aug 24 09:53:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:43.044944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3130-71ae-4f73-bc0e-afbe8c4e4548 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.045271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3130-71ae-4f73-bc0e-afbe8c4e4548 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.045559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b3130-71ae-4f73-bc0e-afbe8c4e4548 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.046952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 725/1412] 10.222.0.78 () {54 vars in 918 bytes} [Wed Aug 24 09:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:43.054214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042f096-6501-4534-9b90-732d93cda44c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:43.054738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042f096-6501-4534-9b90-732d93cda44c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:43.055368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042f096-6501-4534-9b90-732d93cda44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:43.149163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:43.152619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5042f096-6501-4534-9b90-732d93cda44c None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:43.154659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5042f096-6501-4534-9b90-732d93cda44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:43.311253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 688/1413] 10.222.0.78 () {58 vars in 1015 bytes} [Wed Aug 24 09:53:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:45.807222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e40764-710c-46ae-8885-709709651ca2 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.807832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e40764-710c-46ae-8885-709709651ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.807832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e40764-710c-46ae-8885-709709651ca2 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.809129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 726/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:45.815158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cb32-e425-4cb5-9f28-36ad65aa9ecb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:45.815706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cb32-e425-4cb5-9f28-36ad65aa9ecb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:45.815706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580cb32-e425-4cb5-9f28-36ad65aa9ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:45.899983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3580cb32-e425-4cb5-9f28-36ad65aa9ecb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:45.901238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3580cb32-e425-4cb5-9f28-36ad65aa9ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:45.973139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:46.024457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 689/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:46.333863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e40764-710c-46ae-8885-709709651ca2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:46.335493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-22e40764-710c-46ae-8885-709709651ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:46.520813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e40764-710c-46ae-8885-709709651ca2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:46.606579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:46.656894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:46.657421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:46.657700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:46.657968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:46.659656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:46.770187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:46.772468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633088fd-d4c1-48c1-9a90-e0285f3abccc service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:46.796117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 727/1416] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:47.766620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.767155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.767155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.768575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 690/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:47.774511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf778-d4ef-4457-88aa-1bb132812231 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:47.775095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf778-d4ef-4457-88aa-1bb132812231 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:47.775095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdf778-d4ef-4457-88aa-1bb132812231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:47.874763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-56fdf778-d4ef-4457-88aa-1bb132812231 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:47.876140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-56fdf778-d4ef-4457-88aa-1bb132812231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:47.907272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:47.990112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 728/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:48.147684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:48.149354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:48.333513 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf7ab7-1a9d-4a1f-9023-03c140d1297b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:48.378497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:48.461090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:48.461713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:48.461988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:48.462259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:48.463993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:48.571754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:48.573466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f422c5-7833-43cf-b515-806aba487efe service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:48.596400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 691/1419] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:49.548162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:49.548780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:49.548780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:49.550149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 729/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:49.556536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be489c-d946-4d05-8ac0-c3ed0492cffd None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:49.557095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be489c-d946-4d05-8ac0-c3ed0492cffd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:49.557095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be489c-d946-4d05-8ac0-c3ed0492cffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:49.620974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:49.648745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-19be489c-d946-4d05-8ac0-c3ed0492cffd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:49.650086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-19be489c-d946-4d05-8ac0-c3ed0492cffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:49.783248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 692/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:49.939869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:49.941418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:50.115781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04fce7-c686-4dae-8afd-6be45f0d4807 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:50.129202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:50.247543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:50.248195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:50.248475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:50.248749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:50.250401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:50.353993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:50.355628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8120da09-9449-453e-897e-b8e5b9fdc92d service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:50.377008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 730/1422] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:51.290120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.290716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.290716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.292089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 693/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:51.297148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e984-c5c1-4d22-a4b2-786ac3e6d824 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.297931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e984-c5c1-4d22-a4b2-786ac3e6d824 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.297931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7e984-c5c1-4d22-a4b2-786ac3e6d824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.318204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:51.377243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3ed7e984-c5c1-4d22-a4b2-786ac3e6d824 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:51.378485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3ed7e984-c5c1-4d22-a4b2-786ac3e6d824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:51.491194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 731/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:51.646635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:51.648232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:51.766462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:51.806066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ab90b-1cd9-400e-8c4a-f0dde11a3a9c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:51.925419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:51.925930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:51.926206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:51.927684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:51.929365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:52.010608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:52.025796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:52.027462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1da887-e3ca-40fb-a733-3d56112b9c65 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:52.047862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 694/1425] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:52.945388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.945983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.945983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:52.947375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 732/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:52.952801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211067e-4048-4ff2-aa02-dacad45c4c92 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:52.953315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211067e-4048-4ff2-aa02-dacad45c4c92 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:52.953423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211067e-4048-4ff2-aa02-dacad45c4c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:53.043714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b211067e-4048-4ff2-aa02-dacad45c4c92 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:53.044994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b211067e-4048-4ff2-aa02-dacad45c4c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:53.162103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 695/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:53.320203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:53.321825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:53.413128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:53.482890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb40ea-cbc7-435d-a381-a944c0de9116 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:53.631105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:53.631932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:53.632368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:53.632785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:53.635467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:53.708865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:53.754166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:53.755976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03454d2e-2f29-4ef5-9dbe-43037f1aea16 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:53.774752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 733/1428] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:54.596448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:54.597298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:54.597298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:54.599323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 696/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:54.605143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a1ff5-5b73-4220-ba21-7d3c3c88500f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:54.605643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a1ff5-5b73-4220-ba21-7d3c3c88500f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:54.605754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a1ff5-5b73-4220-ba21-7d3c3c88500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:54.692811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0a4a1ff5-5b73-4220-ba21-7d3c3c88500f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:54.694060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0a4a1ff5-5b73-4220-ba21-7d3c3c88500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:54.797970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 734/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:54.956015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:54.958364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:55.012359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:55.117458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc32e-a375-4b1c-8b92-adabc177bdcb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:55.237580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:55.238198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:55.239122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:55.239454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:55.241188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:55.248625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:55.348317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:55.349953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b324cc-803a-44a0-9aa3-db8fa51f0183 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:55.369853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 697/1431] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:56.190965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:56.191558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:56.191558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:56.192908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 735/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:56.198672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06ae6-bee2-4bcc-bf9e-b4cc0fd5fd4b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:56.199169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06ae6-bee2-4bcc-bf9e-b4cc0fd5fd4b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:56.199274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb06ae6-bee2-4bcc-bf9e-b4cc0fd5fd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:56.281320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5eb06ae6-bee2-4bcc-bf9e-b4cc0fd5fd4b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:56.282544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5eb06ae6-bee2-4bcc-bf9e-b4cc0fd5fd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:56.355048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:56.386662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 698/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:56.540696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:56.542382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:56.560655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:56.705321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2849778-e114-4e8a-9cb0-d2fe406530a5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:56.798175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:56.826207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:56.826814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:56.827093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:56.827364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:56.829062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:56.930873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:56.932572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088068f-9715-49d2-951e-f8b38b8e1674 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:56.951328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 736/1434] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:57.789421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:57.789957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:57.789957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:57.791237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 699/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:57.796602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e8cb-5c1f-4dfd-85e1-fd6d3a4e67e0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:57.797125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e8cb-5c1f-4dfd-85e1-fd6d3a4e67e0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:57.797125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9661e8cb-5c1f-4dfd-85e1-fd6d3a4e67e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:57.877396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9661e8cb-5c1f-4dfd-85e1-fd6d3a4e67e0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:57.878622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9661e8cb-5c1f-4dfd-85e1-fd6d3a4e67e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:57.920722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:57.978389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 737/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:58.130249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:58.131803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:58.286051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f5bd4-b5d5-4581-ac04-3a5bca736182 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:58.343130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:58.413399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:58.414018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:58.414295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:58.414564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:58.416263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:58.510545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:53:58.512208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b98d2-3664-4b17-b4b3-403e5c0f143f service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:53:58.531278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 700/1437] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:53:59.360694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:59.361253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:59.361253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:59.362622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 738/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:53:59.368555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e5ef6-cbef-475c-a34b-fb523d5b452b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:59.369105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e5ef6-cbef-475c-a34b-fb523d5b452b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:59.369105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e5ef6-cbef-475c-a34b-fb523d5b452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:59.451577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-299e5ef6-cbef-475c-a34b-fb523d5b452b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:53:59.452906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-299e5ef6-cbef-475c-a34b-fb523d5b452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:59.455038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:59.555709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 701/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:53:59.715510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:53:59.715510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:53:59.868963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a24d49-3a75-452c-a7af-a9a023574cc5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:53:59.897806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:53:59.993007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:53:59.993522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:53:59.993799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:53:59.994070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:53:59.995695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:00.087942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:00.089572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc19912-8b70-4d8e-9202-e060a04d4807 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:00.108246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 739/1440] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:54:01.954253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:01.954876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:01.954876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:01.956191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 702/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:01.962388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8ab0-3814-479d-bf9e-c532bcb95309 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:01.962888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8ab0-3814-479d-bf9e-c532bcb95309 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:01.962985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e8ab0-3814-479d-bf9e-c532bcb95309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:02.007443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:02.035318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3c1e8ab0-3814-479d-bf9e-c532bcb95309 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:02.036662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3c1e8ab0-3814-479d-bf9e-c532bcb95309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:02.142713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 740/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:02.430139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:02.431760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:02.542863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4944c4-68bc-4f48-9e97-2f01c944e28f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:02.575716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:02.621285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:02.621913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:02.622191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:02.622459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:02.624217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:02.741044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:02.742505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3f857-03d7-467a-9d6b-40d13e62a90f service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:02.761411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 703/1443] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:04.913605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:04.914199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:04.914199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:04.915501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 741/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:04.921844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8de3-4003-47ca-9cbb-3ab1677c5803 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:04.922352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8de3-4003-47ca-9cbb-3ab1677c5803 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:04.922448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032e8de3-4003-47ca-9cbb-3ab1677c5803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:05.008624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-032e8de3-4003-47ca-9cbb-3ab1677c5803 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:05.010051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-032e8de3-4003-47ca-9cbb-3ab1677c5803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:05.025735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:05.119195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 704/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:05.404553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:05.406177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:05.515868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d633b12-5ba7-4686-8bed-1b627a29ba56 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:05.528927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:05.594861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:05.595465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:05.595743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:05.596075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:05.598365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:05.714118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:05.715476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0cb2af-7b0c-476d-bd53-a7f3b2946dd3 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:05.734998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 742/1446] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:08.011570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:08.012204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:08.012204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:08.013520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 705/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:08.019694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98576-c34e-48bc-8158-ed325254528b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:08.020208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98576-c34e-48bc-8158-ed325254528b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:08.020319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d98576-c34e-48bc-8158-ed325254528b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:08.070016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:08.088404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-59d98576-c34e-48bc-8158-ed325254528b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:08.089650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-59d98576-c34e-48bc-8158-ed325254528b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:08.190047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 743/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:54:08.487779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:08.489373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:08.597199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28f6a1-fb6f-4b09-8b7f-60ecf87d6ec6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:08.641195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:08.679180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:08.679817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:08.680115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:08.680391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:08.682033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:08.795270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:08.796678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ece2d-ce1d-4c7f-af74-2b09463b6268 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:08.817762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 706/1449] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:10.931904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e544-51ca-4c31-a101-6c133779467c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:10.932471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e544-51ca-4c31-a101-6c133779467c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:10.932471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377e544-51ca-4c31-a101-6c133779467c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:10.933837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 744/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:10.939571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d029-d54d-4edf-bcd8-07b380376e5a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:10.940628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d029-d54d-4edf-bcd8-07b380376e5a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:10.940847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047d029-d54d-4edf-bcd8-07b380376e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:11.009702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a047d029-d54d-4edf-bcd8-07b380376e5a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:11.010941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a047d029-d54d-4edf-bcd8-07b380376e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:11.032410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:11.115818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 707/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:11.404455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e544-51ca-4c31-a101-6c133779467c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:11.406007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2377e544-51ca-4c31-a101-6c133779467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:11.516569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377e544-51ca-4c31-a101-6c133779467c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:11.538004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:11.594535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:11.595150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:11.595431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:11.595708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:11.597891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:11.720930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:11.722973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400f2c8-323b-47b5-b61f-50f040a619b5 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:11.742656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 745/1452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:13.839869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:13.840423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:13.840423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:13.841988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 708/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:13.848964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ad30-aae6-446d-9bcb-c0bbcb948945 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:13.849493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ad30-aae6-446d-9bcb-c0bbcb948945 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:13.849587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392ad30-aae6-446d-9bcb-c0bbcb948945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:13.917364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:13.920222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1392ad30-aae6-446d-9bcb-c0bbcb948945 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:13.921992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1392ad30-aae6-446d-9bcb-c0bbcb948945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:14.023269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 746/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:54:14.312798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:14.314365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:14.424502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d88eb9-727e-4c99-aacb-dc6c1b9c85ac None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:14.476146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:14.503636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:14.504287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:14.504567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:14.504842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:14.506476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:14.618822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:14.620297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46d6327-fe15-4557-b039-eca8b03f63d6 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:14.639273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 709/1455] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:16.672397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:16.672940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:16.673067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:16.674358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 747/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:16.680045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739caef-c2a8-4199-ac19-f8c85d0ae1e8 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:16.680509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739caef-c2a8-4199-ac19-f8c85d0ae1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:16.680608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739caef-c2a8-4199-ac19-f8c85d0ae1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:16.749912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6739caef-c2a8-4199-ac19-f8c85d0ae1e8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:16.751063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6739caef-c2a8-4199-ac19-f8c85d0ae1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:16.782115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:16.852999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 710/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:17.139369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:17.140981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:17.248687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e983-6af8-4d45-866f-595025ff5bfa None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:17.277913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:17.325678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:17.326291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:17.326572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:17.326845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:17.328494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:17.442554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:17.444049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac9e6e-fb87-4140-ad22-771b82338386 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:17.465179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 748/1458] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:19.477455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:19.477989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:19.477989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:19.479382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 711/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:19.484645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b91c64-7c07-4c4c-a716-ba57cbfa1eae None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:19.485054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b91c64-7c07-4c4c-a716-ba57cbfa1eae None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:19.485204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b91c64-7c07-4c4c-a716-ba57cbfa1eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:19.553878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-40b91c64-7c07-4c4c-a716-ba57cbfa1eae None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:19.555110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-40b91c64-7c07-4c4c-a716-ba57cbfa1eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:19.563747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:19.659771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 749/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:19.947402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:19.948998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:20.054676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3004afcf-1a2d-47bb-8af3-4b9f7bdbb010 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:20.119298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:20.137743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:20.138463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:20.138744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:20.139016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:20.140665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:20.263818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:20.265169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69f0bd-8967-4573-b40f-05fcb86eaa18 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:20.284590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 712/1461] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:22.294321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:22.295112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:22.295298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:22.297757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 750/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:22.304445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f4a4-b9e4-4e67-8d2f-32501bd6da5c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:22.306199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f4a4-b9e4-4e67-8d2f-32501bd6da5c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:22.306199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2f4a4-b9e4-4e67-8d2f-32501bd6da5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:22.393361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5cf2f4a4-b9e4-4e67-8d2f-32501bd6da5c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:22.394620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5cf2f4a4-b9e4-4e67-8d2f-32501bd6da5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:22.434338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:22.498655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 713/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:22.803322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:22.804942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:23.032617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a7e8e-e77a-4075-8493-c49c433a510f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:23.073804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:23.113436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:23.114041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:23.114315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:23.114580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:23.116219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:23.228250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:23.229712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f546c8-0d0d-4e24-9c82-06072f793e5e service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:23.248589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 751/1464] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:25.252072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:25.252868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:25.252982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:25.254362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 714/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:25.259881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc0d1c-465f-4ce5-bcac-a8a234e04880 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:25.260357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc0d1c-465f-4ce5-bcac-a8a234e04880 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:25.260465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc0d1c-465f-4ce5-bcac-a8a234e04880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:25.328376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-67dc0d1c-465f-4ce5-bcac-a8a234e04880 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:25.330287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-67dc0d1c-465f-4ce5-bcac-a8a234e04880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:25.348803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:25.429894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 752/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:25.717857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:25.719480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:25.824812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83160a26-9438-48c2-bf21-fb03f9ffcde7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:25.897342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:25.905036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:25.905664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:25.905943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:25.906212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:25.908244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:26.024412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:26.025762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097db86-9675-4ecd-a831-2a33ace4c733 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:26.044288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 715/1467] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:28.049270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:28.049871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:28.049984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:28.051264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 753/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:28.056586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8081b-5dcf-41c0-a8fd-cc54a8892eb5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:28.056993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8081b-5dcf-41c0-a8fd-cc54a8892eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:28.057140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8081b-5dcf-41c0-a8fd-cc54a8892eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:28.123883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-76f8081b-5dcf-41c0-a8fd-cc54a8892eb5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:28.125103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-76f8081b-5dcf-41c0-a8fd-cc54a8892eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:28.175380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:28.343191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 716/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:28.634127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:28.635689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:28.742791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad70157-e4ce-4e33-a211-7a485ebb8a6a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:28.791113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:28.817432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:28.818039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:28.818320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:28.818594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:28.820300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:28.931847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:28.933287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8e8b2-5615-48e8-8aca-cb5b5a3f60a1 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:28.953070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 754/1470] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:38.341277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=22 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:38.950725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=66 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:49.098588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631df6-3f45-4e33-bac1-8c2916eba12c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:49.099157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631df6-3f45-4e33-bac1-8c2916eba12c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:49.099157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac631df6-3f45-4e33-bac1-8c2916eba12c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:49.100525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 717/1471] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:49.105763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2739-26de-4c73-8967-1f62d78beff9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:49.106177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2739-26de-4c73-8967-1f62d78beff9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:49.106323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd2739-26de-4c73-8967-1f62d78beff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:49.172728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cbdd2739-26de-4c73-8967-1f62d78beff9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:49.173952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cbdd2739-26de-4c73-8967-1f62d78beff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:49.283206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 755/1472] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:49.589689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:49.590105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:49.590375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:49.591723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 718/1473] 10.222.0.78 () {54 vars in 967 bytes} [Wed Aug 24 09:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:49.596832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99168-16e2-41eb-a9bf-d42641f815d4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:49.597115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99168-16e2-41eb-a9bf-d42641f815d4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:49.597384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d99168-16e2-41eb-a9bf-d42641f815d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:49.677053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a0d99168-16e2-41eb-a9bf-d42641f815d4 None None] MFA Rules not processed for user `3061244943c84217bf9ba30224cd4951`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:49.678272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a0d99168-16e2-41eb-a9bf-d42641f815d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:49.699120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:49.759215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 756/1474] 10.222.0.78 () {58 vars in 1064 bytes} [Wed Aug 24 09:54:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:49.767082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:49.768656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:49.874035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234d8990-3592-434f-984f-a5aa0c89b57e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:49.950224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:49.950922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:49.951395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:49.951605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:49.953715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:50.009058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:50.066358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:50.067819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460717e-24a8-491b-967e-0cc34099b21d service glance] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:50.086780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 719/1475] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:52.263634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:52.264239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:52.264239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:52.266380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 757/1476] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:52.271419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182cf4-6687-43cb-8131-6a6a0a134a9f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:52.271891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182cf4-6687-43cb-8131-6a6a0a134a9f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:52.272001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d182cf4-6687-43cb-8131-6a6a0a134a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:52.337909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3d182cf4-6687-43cb-8131-6a6a0a134a9f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:52.339167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3d182cf4-6687-43cb-8131-6a6a0a134a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:52.437930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 720/1477] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:52.713619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:52.715359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:52.841608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30fe0a1-bfc6-43b3-afdb-d2b70dc4d2f0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:52.891771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:52.938195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:52.938806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:52.939082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:52.939351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:52.940035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:52.941852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:54:52.941852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:54:52.941852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:54:52.942988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb335651-e3e9-4f3e-8dc0-d69f74fc9689 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:52.962778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 758/1478] 10.222.0.78 () {56 vars in 1108 bytes} [Wed Aug 24 09:54:52 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:54:54.920871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:54.921605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:54.921605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:54.922836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 721/1479] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:54:54.928498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debb71-3c14-4160-a693-8168baa3907a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:54.928981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debb71-3c14-4160-a693-8168baa3907a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:54.929083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8debb71-3c14-4160-a693-8168baa3907a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:55.003837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d8debb71-3c14-4160-a693-8168baa3907a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:55.005062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d8debb71-3c14-4160-a693-8168baa3907a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:55.104619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 759/1480] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:55.404016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:55.405567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:55.415540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:55.564348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a07fe5-3a10-4e4c-82a8-4f82c8a151d6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:55.650857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:55.685260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:55.685991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:55.686273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:55.686673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:55.689527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:55.783418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:55.785043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e32a1-85ea-423b-b89d-519e5bd83f6d service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:55.804238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 722/1481] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:54:57.924537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:57.925113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:57.925113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:57.926456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 760/1482] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:54:57.932061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a0fc9-5885-4f13-94d4-44a75d4d9117 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:57.932573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a0fc9-5885-4f13-94d4-44a75d4d9117 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:57.932676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a0fc9-5885-4f13-94d4-44a75d4d9117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:57.999296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-433a0fc9-5885-4f13-94d4-44a75d4d9117 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:54:58.000875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-433a0fc9-5885-4f13-94d4-44a75d4d9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:58.048494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:58.099464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 723/1483] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:54:58.394300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:54:58.395891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:58.421253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:58.507156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63998a-98bc-4bf2-ae87-c89eadff55a2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:54:58.583378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:54:58.584014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:54:58.584295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:54:58.584568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:54:58.586188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:54:58.662883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:54:58.700031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:54:58.701475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5e426-e590-4c98-8ac4-e28895c7e28e service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:54:58.721165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 761/1484] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:00.745083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:00.745645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:00.745645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:00.746913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 724/1485] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:00.752745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b26b5-795c-487b-8be3-661ee0b6e0e7 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:00.753185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b26b5-795c-487b-8be3-661ee0b6e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:00.753297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b26b5-795c-487b-8be3-661ee0b6e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:00.819663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-da8b26b5-795c-487b-8be3-661ee0b6e0e7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:00.820906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-da8b26b5-795c-487b-8be3-661ee0b6e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:00.919025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 762/1486] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:01.208169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:01.209751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:01.322127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d785f-30df-4a67-b638-f69e5bd13dac None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:01.402529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:01.403537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:01.404182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:01.404462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:01.404805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:01.407586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:01.521731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:01.523188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46086a7f-ff6d-48c8-a1c5-c5ec8c2e93dd service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:01.542547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 725/1487] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:03.725076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:03.725645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:03.725645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:03.726890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 763/1488] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:03.732335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d2b1-5404-406f-9957-33fabb116fc6 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:03.732778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d2b1-5404-406f-9957-33fabb116fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:03.732891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912d2b1-5404-406f-9957-33fabb116fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:03.798649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f912d2b1-5404-406f-9957-33fabb116fc6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:03.799913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f912d2b1-5404-406f-9957-33fabb116fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:03.853679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:03.900366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 726/1489] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:55:04.194200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:04.195750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:04.235187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:04.313909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9534e54-d23a-4046-8dc4-5a9a46063db9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:04.391957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:04.392571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:04.392849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:04.393119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:04.394737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:04.479740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:04.510374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:04.511847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ab85f-0b76-4c97-bf52-992f7b5d01bc service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:04.532150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 764/1490] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:06.659588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:06.660274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:06.660274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:06.661756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 727/1491] 10.222.0.78 () {54 vars in 927 bytes} [Wed Aug 24 09:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:55:06.667483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660344c-09ad-4bf0-a4b0-8f5d7e8872f5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:06.667952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660344c-09ad-4bf0-a4b0-8f5d7e8872f5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:06.668074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0660344c-09ad-4bf0-a4b0-8f5d7e8872f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:06.738635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0660344c-09ad-4bf0-a4b0-8f5d7e8872f5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:55:06.739897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0660344c-09ad-4bf0-a4b0-8f5d7e8872f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:06.844460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 765/1492] 10.222.0.78 () {58 vars in 1024 bytes} [Wed Aug 24 09:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:55:07.157382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:55:07.158956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:07.268691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b73a1f-b1f1-4dba-9ffe-11561d810992 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:55:07.351175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:55:07.351806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:55:07.352087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:55:07.352359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:55:07.353994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:55:07.364448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:07.475976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:55:07.477428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62832057-5430-4ba0-9445-bae5575b4ad2 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:55:07.496379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 728/1493] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:55:16.841716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=82 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:55:17.494285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=50 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:20.646227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:20.646837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:20.646837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:20.728106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:20.729266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:20.836634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 766/1494] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:20.845446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:20.846019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:20.846019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:20.935254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:20.937358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:21.037642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 729/1495] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:21.042283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.043503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:21.055691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5293a-8f77-487e-8b77-315d8ddf679e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:21.151812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:21.193562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:21.194184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:21.194458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:21.194728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:21.195399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:56:21.195695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:21.196778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:21.196778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:56:21.196778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:56:21.197685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:21.210706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 767/1496] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:56:21.219644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.229473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c07010-9287-49f8-ab64-c2f63e0bb5dd None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:21.298809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:21.338791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:21.339432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:21.339715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:21.340023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:21.341089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:21.343349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:21.343349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:56:21.343349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:56:21.344607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:21.409810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 730/1497] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:56:21 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:21.414065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.415599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:21.468919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1dab06-ed77-4b74-a460-be00be81788f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:21.581816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:21.582339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:21.582630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:21.582968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:21.583517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:21.584578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:21.584578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:56:21.584578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:56:21.585818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:21.631166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 768/1498] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:56:21 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:56:21.635505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.637414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:21.703700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f050564-a0d6-4ad6-a352-993d676ff6cc admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:21.805042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:21.820543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:21.821188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:21.821478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:21.821747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:21.822384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:21.823403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:21.823403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:56:21.823403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:56:21.824376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:21.837970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 731/1499] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:56:21.843028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.844603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:21.856115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc2d75-6392-4c1e-9cd9-a92ffc450e81 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:21.882612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:21.961826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:21.962443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:21.962720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:21.962988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:21.963639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:21.964707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:21.964707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:56:21.964707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:56:21.965612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2b1752-455e-45c1-aba5-ef9649483cd3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:21.979979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 769/1500] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:56:21.985212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:21.986779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:22.024847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74898934-e76d-4b3d-930b-9d58719a8061 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:22.134838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:22.135797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.136106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.136385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] PATH_INFO: `/v3/projects/b31181d5417d40f89c343525a7954866/users/af94d92f508f40b59f44b7a87eb8d9e2/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:22.164052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31181d5417d40f89c343525a7954866, user_id=af94d92f508f40b59f44b7a87eb8d9e2, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:22.165098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:22.165098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:56:22.165098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:56:22.167064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910d0e3-f481-4808-8df8-d76d4647a20b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:22.199343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 732/1501] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:56:21 2022] PUT /identity/v3/projects/b31181d5417d40f89c343525a7954866/users/af94d92f508f40b59f44b7a87eb8d9e2/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:22.213108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3b8-b746-4a99-a42c-44f7e9ada18a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.213399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3b8-b746-4a99-a42c-44f7e9ada18a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.213676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbfd3b8-b746-4a99-a42c-44f7e9ada18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:22.275841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7dbfd3b8-b746-4a99-a42c-44f7e9ada18a None None] MFA Rules not processed for user `af94d92f508f40b59f44b7a87eb8d9e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:22.277083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7dbfd3b8-b746-4a99-a42c-44f7e9ada18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:22.371597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 770/1502] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:22.385404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.385615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.385911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:22.387291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 733/1503] 10.222.0.78 () {54 vars in 964 bytes} [Wed Aug 24 09:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:22.393123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639a93-ec1d-4122-9bcd-4909a7489749 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.393406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639a93-ec1d-4122-9bcd-4909a7489749 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.393725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2639a93-ec1d-4122-9bcd-4909a7489749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:22.427548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:22.482421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a2639a93-ec1d-4122-9bcd-4909a7489749 None None] MFA Rules not processed for user `6b171220975d48488be864348cc9aca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:22.484128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a2639a93-ec1d-4122-9bcd-4909a7489749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:22.579561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 771/1504] 10.222.0.78 () {58 vars in 1061 bytes} [Wed Aug 24 09:56:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:56:22.587467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:22.589058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:22.652940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:22.717841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba305499-b332-47a6-8468-23c044e29f2e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:22.803150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:22.804057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.804468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.804871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:22.807483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:22.929014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:22.930698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:22.934801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:22.950944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 734/1505] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:22.998279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eea03a-a343-458c-9ea6-458fc5c8d6f4 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:22.999430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eea03a-a343-458c-9ea6-458fc5c8d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:22.999543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eea03a-a343-458c-9ea6-458fc5c8d6f4 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.000986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 772/1506] 10.222.0.78 () {52 vars in 830 bytes} [Wed Aug 24 09:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:56:23.004944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:23.006170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.114929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413f18f-a1ce-432a-9d79-a7b5d88aedc1 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:23.219733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:23.220325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.220603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.220872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] PATH_INFO: `/v3/users/af94d92f508f40b59f44b7a87eb8d9e2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.234456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af94d92f508f40b59f44b7a87eb8d9e2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:23.235496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:23.235496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:56:23.235496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:56:23.236780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:23.247517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:23.316058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af94d92f508f40b59f44b7a87eb8d9e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:23.316510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af94d92f508f40b59f44b7a87eb8d9e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:23.342113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af94d92f508f40b59f44b7a87eb8d9e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:23.353781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af94d92f508f40b59f44b7a87eb8d9e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:23.370313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af94d92f508f40b59f44b7a87eb8d9e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:23.376480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 735/1507] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:56:23 2022] DELETE /identity/v3/users/af94d92f508f40b59f44b7a87eb8d9e2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:56:23.381112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.381482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.381766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.444912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:56:23.446151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.534650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:23.549924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 773/1508] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:56:23.561534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:23.562787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.677251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426c4cf9-817a-44fa-aa25-b32bde778ca3 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:23.749109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:23.771499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:23.772147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:23.772425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:23.772697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:23.774325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:23.894006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:23.895496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0682ba-2a7f-4866-9bc1-b75006f0d269 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:23.915902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 736/1509] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:56:24.322018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:56:24.323358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:56:24.337564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb248398-1602-4a88-a501-41bbda9bc8fc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:56:24.447971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:56:24.448880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:56:24.449342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:56:24.449820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] PATH_INFO: `/v3/projects/b31181d5417d40f89c343525a7954866` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:56:24.463476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31181d5417d40f89c343525a7954866)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:56:24.464575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:56:24.464575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:56:24.464575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:56:24.465802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:56:24.530899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31181d5417d40f89c343525a7954866', 'request_id': 'req-de341b0d-d2b3-425b-a849-5153adbd7dce'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:24.541138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31181d5417d40f89c343525a7954866', 'request_id': 'req-de341b0d-d2b3-425b-a849-5153adbd7dce'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:24.558470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31181d5417d40f89c343525a7954866', 'request_id': 'req-de341b0d-d2b3-425b-a849-5153adbd7dce'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:24.586618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] The token cache is being invalidate because project b31181d5417d40f89c343525a7954866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:56:24.586966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:24.587355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-de341b0d-d2b3-425b-a849-5153adbd7dce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:56:24.589430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 774/1510] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 09:56:24 2022] DELETE /identity/v3/projects/b31181d5417d40f89c343525a7954866 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:56:33.913635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=62 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:56:34.583748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:55.389941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.390559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.390559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.391864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 737/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:55.397370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92f71b-31a1-46db-a236-da03f0e0e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.397818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92f71b-31a1-46db-a236-da03f0e0e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.397916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92f71b-31a1-46db-a236-da03f0e0e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.480329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2c92f71b-31a1-46db-a236-da03f0e0e0c6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:55.481554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2c92f71b-31a1-46db-a236-da03f0e0e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:55.588427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 775/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:55.747053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:55.749342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:55.870804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8338b-f125-4536-b5ba-30c91d6bc4a2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:55.877219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:55.878687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:55.879094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:55.879372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:55.887542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:55.889170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e16f1f-da62-4ffb-93c1-93d77f6a4125 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:55.898875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 738/1513] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:56.732846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf794c-a1d4-4537-8ffd-0afafb36a207 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.733428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf794c-a1d4-4537-8ffd-0afafb36a207 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.733428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf794c-a1d4-4537-8ffd-0afafb36a207 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.734759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 776/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:57:56.739773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7f5a1-7049-4910-afc6-28f85f06d631 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.740238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7f5a1-7049-4910-afc6-28f85f06d631 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.740398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7f5a1-7049-4910-afc6-28f85f06d631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.759512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-50f7f5a1-7049-4910-afc6-28f85f06d631 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:56.760757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-50f7f5a1-7049-4910-afc6-28f85f06d631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:56.769238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 739/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:57:56.921925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf794c-a1d4-4537-8ffd-0afafb36a207 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:57:56.936135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf794c-a1d4-4537-8ffd-0afafb36a207 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:57:56.942300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:57:56.942885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:56.943159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:56.943426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:56.951241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:57:56.952856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eb494-cf70-4d78-8a93-6e52c6135ca9 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:57:56.962802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 777/1516] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:57:57.114303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbb42a-8e8a-43f1-97bb-70bb62360d00 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.114583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbb42a-8e8a-43f1-97bb-70bb62360d00 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.114863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cbb42a-8e8a-43f1-97bb-70bb62360d00 None None] PATH_INFO: `/v3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.116180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 740/1517] 10.222.0.84 () {54 vars in 925 bytes} [Wed Aug 24 09:57:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:57:57.122969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff154-ce7e-4cf3-8967-8b3364d17296 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:57:57.123253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff154-ce7e-4cf3-8967-8b3364d17296 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:57:57.123525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ff154-ce7e-4cf3-8967-8b3364d17296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:57:57.182905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-690ff154-ce7e-4cf3-8967-8b3364d17296 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:57:57.184168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-690ff154-ce7e-4cf3-8967-8b3364d17296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:57:57.200613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:57:57.256343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 778/1518] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 09:57:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:06.434227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36215b-9dc9-4613-9a9b-3ca4ad0be442 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.434794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36215b-9dc9-4613-9a9b-3ca4ad0be442 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.434794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a36215b-9dc9-4613-9a9b-3ca4ad0be442 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.436048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 741/1519] 10.222.0.78 () {54 vars in 961 bytes} [Wed Aug 24 09:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:06.441006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf773f3-2845-4401-8e41-1f84ecce315b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.441445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf773f3-2845-4401-8e41-1f84ecce315b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.441550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf773f3-2845-4401-8e41-1f84ecce315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.462860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bdf773f3-2845-4401-8e41-1f84ecce315b None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:06.464097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bdf773f3-2845-4401-8e41-1f84ecce315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:06.472910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 779/1520] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Aug 24 09:58:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:06.481072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36215b-9dc9-4613-9a9b-3ca4ad0be442 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:06.494390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a36215b-9dc9-4613-9a9b-3ca4ad0be442 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:06.499249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:06.499862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:06.500143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:06.500412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:06.509460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:06.511068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b302d7-62cc-4ef9-9ed9-37209d230c8e service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:06.520747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 742/1521] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:58:08.188474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fafbe2-dffa-47d8-8867-beed7710f6cb None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.189058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fafbe2-dffa-47d8-8867-beed7710f6cb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.189058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fafbe2-dffa-47d8-8867-beed7710f6cb None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.190377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 780/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:08.196485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2d11-6488-4a5c-97c1-e20c45002080 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.196990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2d11-6488-4a5c-97c1-e20c45002080 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.197109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2d11-6488-4a5c-97c1-e20c45002080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.217756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c82c2d11-6488-4a5c-97c1-e20c45002080 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:08.218971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c82c2d11-6488-4a5c-97c1-e20c45002080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:08.227893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 743/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:08.381740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fafbe2-dffa-47d8-8867-beed7710f6cb None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:08.397870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fafbe2-dffa-47d8-8867-beed7710f6cb None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:08.403319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:08.403952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:08.404228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:08.404478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:08.413677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:08.415276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69578841-fc22-4483-b1d1-30549126d27d service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:08.425145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 781/1524] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:09.295392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946e0b-4649-4a17-8f37-9f1f6fbedf3f None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.295958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946e0b-4649-4a17-8f37-9f1f6fbedf3f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.295958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42946e0b-4649-4a17-8f37-9f1f6fbedf3f None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.297322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 744/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 09:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:09.302305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c8e9-34c8-4e9b-955b-9605377219aa None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.302784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c8e9-34c8-4e9b-955b-9605377219aa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.302880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5c8e9-34c8-4e9b-955b-9605377219aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.323376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-dbd5c8e9-34c8-4e9b-955b-9605377219aa None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:09.324626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-dbd5c8e9-34c8-4e9b-955b-9605377219aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:09.332551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 782/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 09:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:09.497089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42946e0b-4649-4a17-8f37-9f1f6fbedf3f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:09.510662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42946e0b-4649-4a17-8f37-9f1f6fbedf3f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:09.516112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:09.516715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:09.516991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:09.517260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:09.527268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:09.528901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fdc98e-a624-49d5-8e29-8e6fff1737f2 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:09.540065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 745/1527] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:14.430716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.431323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.431323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.432680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 783/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 09:58:14.438377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f21ca-650b-4bb7-9b0a-4440537e413c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.438854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f21ca-650b-4bb7-9b0a-4440537e413c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.438968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8f21ca-650b-4bb7-9b0a-4440537e413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.459180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2a8f21ca-650b-4bb7-9b0a-4440537e413c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:14.461127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2a8f21ca-650b-4bb7-9b0a-4440537e413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:14.469328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 746/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:14.755628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:14.757981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:14.840990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61573-d972-484a-83ae-0393dc57ae6e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:14.844575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:14.845182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:14.845466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:14.845730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:14.856786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:14.858234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5beaf8-a69d-4a7d-9418-9d60ed019b95 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:14.869098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 784/1530] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:16.820857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.821439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.821439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.822780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 747/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:16.828504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96588-af2d-44a4-8b97-8d7858a4e710 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:16.828982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96588-af2d-44a4-8b97-8d7858a4e710 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:16.829079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf96588-af2d-44a4-8b97-8d7858a4e710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:16.849608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fbf96588-af2d-44a4-8b97-8d7858a4e710 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:16.851601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fbf96588-af2d-44a4-8b97-8d7858a4e710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:16.859577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 785/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:17.151712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:17.153456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:17.167906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b71864-ca6d-4d15-b07a-72c84df618a9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:17.171418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:17.172061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:17.172333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:17.172652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:17.184837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:17.186296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97be315-9506-445d-90a2-d0cb8692fc9e service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:17.196417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 748/1533] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:20.855386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.856003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.856003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.857365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 786/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:20.863124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df641ae-8d4c-4e06-8d9e-dcf8d0f65d1c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:20.863637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df641ae-8d4c-4e06-8d9e-dcf8d0f65d1c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:20.863733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df641ae-8d4c-4e06-8d9e-dcf8d0f65d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:20.885244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2df641ae-8d4c-4e06-8d9e-dcf8d0f65d1c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:20.886559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2df641ae-8d4c-4e06-8d9e-dcf8d0f65d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:20.895068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 749/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:21.459180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:21.461212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:21.533582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:21.548632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bcc6c-43df-45dd-9bf4-87c4c4db39a5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:21.554655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:21.555281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:21.555562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:21.555867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:21.565610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:21.567053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ac526-36cc-46c3-8f4b-3e35f9f96795 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:21.577786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 787/1536] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:23.591124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.591706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.591706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.593067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 750/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 09:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:58:23.598769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cf27-c8c3-49a4-97c3-3c7298ae21b3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:23.599288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cf27-c8c3-49a4-97c3-3c7298ae21b3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:23.599288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4cf27-c8c3-49a4-97c3-3c7298ae21b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:23.620432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f3d4cf27-c8c3-49a4-97c3-3c7298ae21b3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:58:23.621661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f3d4cf27-c8c3-49a4-97c3-3c7298ae21b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:23.633448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 788/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 09:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:58:24.185087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:58:24.186792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:58:24.195115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=84 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:24.202066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58269bee-d406-411b-9973-faf6bc8fec42 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:58:24.205660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:58:24.206273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:58:24.206555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:58:24.206826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:58:24.218334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:58:24.220629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45b28f1-a059-4e91-81f6-2aeab8a0fb8d service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:58:24.231850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 751/1539] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:58:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:58:33.614748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=11 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:58:34.227375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:49.999826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.003217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.003217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.028773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.030688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.041909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 789/1540] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:50.052087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.052743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.052743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.085136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.086408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.096631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 752/1541] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.101670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.122965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb99518-50d9-4605-b528-fb13964a92d4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.131201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.132079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.132359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.132635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.133287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:50.133585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.135193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835bee3-2f44-4715-b13e-a2ce283155f8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.153136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 790/1542] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:50.161634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.176051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e8b22-8077-4dec-b726-3a9f78fa3641 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.180256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51171-f401-463e-a315-bf3274bb1647 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.180553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51171-f401-463e-a315-bf3274bb1647 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.181058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c51171-f401-463e-a315-bf3274bb1647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.181423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.182038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.182319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.182593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.183231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.185315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.226229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-29c51171-f401-463e-a315-bf3274bb1647 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.227302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-29c51171-f401-463e-a315-bf3274bb1647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.249466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 791/1543] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:50.258506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.258722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.258965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.259899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 753/1544] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:50.263842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.293369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.294585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.366854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc385f82-8144-4113-bf9a-69700bdb2cce admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.377575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.378217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.378502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.378780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.379442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.380646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:50.380646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:59:50.380646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:59:50.382081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.416258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 792/1545] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:50.422438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.436828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6422dd-aa05-45e8-9e96-fb8ab78d066d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.437268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 754/1546] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/users => generated 284 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:50.450839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.453133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.453744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.454028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.454300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.459822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:50.459822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.459822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7fc754-c561-4b3c-a972-4bf1d9a67d2a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.475445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eae25-237a-4b26-ac88-46ef69ae6e69 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.488061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 793/1547] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 09:59:50.491954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.492876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.493161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.493433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.494102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.495987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.497515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e6f8-165c-4be2-b7c7-07940f4a1206 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.497802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e6f8-165c-4be2-b7c7-07940f4a1206 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.498072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3e6f8-165c-4be2-b7c7-07940f4a1206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.515504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 755/1548] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 09:59:50.516564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.539061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f358f4c-a0ec-46c4-b34d-fb86df3874ee admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.546212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b9f3e6f8-165c-4be2-b7c7-07940f4a1206 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.547433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b9f3e6f8-165c-4be2-b7c7-07940f4a1206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.552323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.552947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.553230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.553507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.554145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.556095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.574582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 794/1549] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.575938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69c82-0b32-4bc9-aa02-586f143c3dcf None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.576261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69c82-0b32-4bc9-aa02-586f143c3dcf None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.577116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb69c82-0b32-4bc9-aa02-586f143c3dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.611781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2eb69c82-0b32-4bc9-aa02-586f143c3dcf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.613396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2eb69c82-0b32-4bc9-aa02-586f143c3dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.625517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 756/1550] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:50.626506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.709906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 795/1551] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.711255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.711548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.711665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da7dbf6-11db-434f-be36-183a0f81592f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.711859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.716380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.717061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.717648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.717932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.718584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.720252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1a621d-77fd-4d30-877a-6b3c751b16c8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.731094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 757/1552] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:50.732023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.732210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde777-8c7a-4d57-9130-12bd7274400b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.732523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde777-8c7a-4d57-9130-12bd7274400b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.732956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddde777-8c7a-4d57-9130-12bd7274400b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.734046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.742108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 796/1553] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.742999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.753968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a75a9-1d63-43e0-aeb2-10341f401ae5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.754515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-eddde777-8c7a-4d57-9130-12bd7274400b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.755738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-eddde777-8c7a-4d57-9130-12bd7274400b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.758538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.759138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.759419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.759689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.760344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.762265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bd47a-e8a5-4c2e-a8fc-59e7406efe73 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.763823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 758/1554] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.765845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860758-e8cd-4db8-859d-684c8abf2cb8 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.766072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860758-e8cd-4db8-859d-684c8abf2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.766344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d860758-e8cd-4db8-859d-684c8abf2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.787321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 797/1555] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/users => generated 291 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:50.787321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4dc45-3a4c-42aa-b982-b1f19da82207 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.787321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4dc45-3a4c-42aa-b982-b1f19da82207 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.787321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc4dc45-3a4c-42aa-b982-b1f19da82207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.788871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-7d860758-e8cd-4db8-859d-684c8abf2cb8 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.790118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7d860758-e8cd-4db8-859d-684c8abf2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.801806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 759/1556] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.803061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.803346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.803617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.805379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bcc4dc45-3a4c-42aa-b982-b1f19da82207 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.806605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bcc4dc45-3a4c-42aa-b982-b1f19da82207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.815441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:50.816873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 798/1557] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.818232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db47d-2f87-46a8-aa43-dd4e48f713f6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.818518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db47d-2f87-46a8-aa43-dd4e48f713f6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.818795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db47d-2f87-46a8-aa43-dd4e48f713f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.833123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.834337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.838665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4e2db47d-2f87-46a8-aa43-dd4e48f713f6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.839905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4e2db47d-2f87-46a8-aa43-dd4e48f713f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.843129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 760/1558] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.843921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.853667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6f26c-3288-45f4-beb9-492f1fd0eae0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.854340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 799/1559] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.855906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.856191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.856460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.861913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.862582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.862890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.863168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] PATH_INFO: `/v3/projects/a3ec5040aac04fe28c38e3add3ca4642/users/97a19ec3aa234a9c8d384086fb5467cd/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.881996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ec5040aac04fe28c38e3add3ca4642, user_id=97a19ec3aa234a9c8d384086fb5467cd, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.883755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:50.885696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:50.886433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.898030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 800/1560] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:50.899049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.904172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 761/1561] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:50 2022] PUT /identity/v3/projects/a3ec5040aac04fe28c38e3add3ca4642/users/97a19ec3aa234a9c8d384086fb5467cd/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:50.905469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:50.919533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd78df4-3015-4057-9aff-2bb531138215 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.929355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:50.972314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ee141-5a5e-4a9b-bf4f-3b1373c8cf82 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:50.976960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.977566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.977861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.978121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.978438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:50.978754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.979072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.979351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.979622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.980561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:50.980684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb349b8d-6d64-4ccb-83f4-4feca1544ecc admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.980862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:50.982473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cf9a0-6c2d-4e69-b5b3-e838c989829e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:50.991196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 762/1562] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:50.992488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.992782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.993060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:50.995150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 801/1563] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:50.996339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f77-70bb-4326-aa78-27822e451dd5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:50.996629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f77-70bb-4326-aa78-27822e451dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:50.996904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f77-70bb-4326-aa78-27822e451dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.018674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f2da7f77-70bb-4326-aa78-27822e451dd5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:51.019397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:51.020093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f2da7f77-70bb-4326-aa78-27822e451dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:51.020629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:51.028132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 802/1564] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:51.028564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 763/1565] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:51.029314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.029452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.029686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.029939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.039475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2ccd8-fddf-43c0-a525-2535e1aa896b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.044054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.044650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.044928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.045761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.047023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:51.047326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.048465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:51.048465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:59:51.048465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:59:51.049353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971cb367-2060-490b-a69d-a99ec658de86 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.049756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:51.051054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:51.061520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 803/1566] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:51.062359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 764/1567] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:51.063262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.063597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.063909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.064202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.075185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc21464b-3858-477b-9fd2-c96efde69266 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.081597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.082219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.082500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.082762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.083450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:51.083781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.085386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.099832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 804/1568] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:51.099832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.100120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] MFA Rules not processed for user `97a19ec3aa234a9c8d384086fb5467cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:51.100430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:51.109809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b9d5d-a445-4775-a51f-cb922306aeb5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.114639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.115371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.115750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.116154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.116923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.118636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.134112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 805/1569] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:51.134112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.142964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5296d88-9455-4c49-91fc-968b390876ff admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.148037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.148628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.148903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.149172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.149919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.150916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:51.150916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:59:51.150916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:59:51.152149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.169639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 765/1570] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:51.171760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.183161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b46adc-dbea-4c32-a1b2-53bd4ef3f502 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.187562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 806/1571] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.187562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.219645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.245340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.245951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.246220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.246488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.247125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:51.247422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.250393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.250912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce553349-69d0-422b-8160-6436fc2bb5da admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.256105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.256722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.257003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.257282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.257943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:51.258250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.260027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.262441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 766/1572] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:51.263323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.276233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 807/1573] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:51.278084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00959114-8ce6-4747-9ea3-68e3753af45f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.284881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.287489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.288498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.288901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.289245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.290110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.292114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.296968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018cbb0-c091-4211-b6bf-e2c39c9312b8 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.301819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.302494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.302868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.303236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.303993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 09:59:51.304430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.306108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.319139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 808/1574] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 09:59:51.320693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.324549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 767/1575] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.324549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.335403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf213a-b7a7-4097-82c1-95d395ab9a9e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.361418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.387760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef76956e-738e-49cc-815e-e4f86f30cdc0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.392588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.393253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.393535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.393821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] PATH_INFO: `/v3/projects/1d18d5eb6d1b4b598f5ed177cdd2f25b/users/122de83caea74b80b6033714f0c88a1b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.397585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.398189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.398466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.398733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.399350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.401290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.409045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d18d5eb6d1b4b598f5ed177cdd2f25b, user_id=122de83caea74b80b6033714f0c88a1b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.411937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.426450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 768/1576] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:51 2022] PUT /identity/v3/projects/1d18d5eb6d1b4b598f5ed177cdd2f25b/users/122de83caea74b80b6033714f0c88a1b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:51.437503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.437642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 809/1577] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.437642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.492156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925bc74-eb1b-4eef-827e-fd24210943ec admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.510068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d321d4a-4740-4408-89fe-f67523147cb7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.515604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.516262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.516544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.516818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.517445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.519380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.544883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.545413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 810/1578] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.545413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.551312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.552339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37602a-8f9e-4276-b884-59198bafce75 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.552554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.552917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.553437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.557042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.557640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.557925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.558200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.558831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.561349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.592364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 811/1579] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.592364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.617559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.619005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.628787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 769/1580] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:51.629368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.639964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5d2e3-322a-478a-ac26-d0ed486a3ab0 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.657813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4eed52-846c-4f4a-8ba9-2be375ca9711 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.663128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.663868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.664225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.664554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.665431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.667236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.699153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 812/1581] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.699991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.705673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.706596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.706732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.707583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.708175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.708967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.711145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.739530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 770/1582] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.740612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.751333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ded56-c9ff-4688-8c00-768ae1a1b835 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.777532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb27c4e-543c-446a-8515-1a3222015e35 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.782591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.783215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.783505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.784367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.784509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.786376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed435d76-ca18-40d8-8e51-9c97c739cfb7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.821706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.822325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.822608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.822985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.823510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.825211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 813/1583] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.826371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.826660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.827053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.828995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.851632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 771/1584] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:51.856012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.858479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] MFA Rules not processed for user `122de83caea74b80b6033714f0c88a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:51.859778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:51.866704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5207e-deac-4996-8f9c-6196f1779b92 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.889882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:51.941480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.942457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.943174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.943392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.944082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.945398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 814/1585] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:51.946594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.946972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.961916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 772/1586] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:51.962076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6006f-66a5-47b9-8fd8-b5c601134698 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.962194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:51.966063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.966844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.967924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.968210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.968836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.971289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6592b71c-169d-4db1-8f03-ca8c9a375436 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:51.971433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:51.975682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:51.976319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:51.976591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:51.976866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:51.977484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:51.979397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.000144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 815/1587] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:52.000144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.006945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 773/1588] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:51 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:52.008002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.008165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811a0fd-4207-49e9-947c-9099783be0fc admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.013746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.014896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.015177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.015629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.016246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.017884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.021170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7a86f-1d8b-4c53-9618-0304f58c86f4 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.025780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.026367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.026649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.026924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.027552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.028849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 816/1589] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.029460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.033174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.039881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d30c2c-c9c2-4b3c-a80e-105672189279 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.044400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.044984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.045292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.045563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.046398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.048010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.059690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 817/1590] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.060580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.064910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 774/1591] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/users => generated 303 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:52.065722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.072280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57610cd-ecc3-4eb7-a495-c5a6faee7f8d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.077791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b168abd-d30f-491e-ac14-00734896a9c0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.078782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.085673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.085673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.085673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.085673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.085673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.092881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.093526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.093833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.094079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.094726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.096458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.111640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 775/1592] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.112731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.130039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085058e-330b-4bdc-8749-5cf8959045d3 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.134800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 818/1593] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:52.135009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.135728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.135954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.136098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.136452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.137078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.138947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.159855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 776/1594] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.159855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.168731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:52.179255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764361b7-721e-4106-93e5-4e49772d3495 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.225549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488755ef-18f5-4ea7-a26b-29424e085ac5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.231458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.232345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.233220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.233890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.234745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.236641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.249132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 819/1595] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.250493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.262341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9600e1-512b-4fba-a7c1-c8ddccd64ce4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.270404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.271231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.271676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.272118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.272992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.273344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.274132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.274669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.275197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.276025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.276202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.278338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.294297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 777/1596] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.294893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 820/1597] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.295962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.296692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.309241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac435760-32db-48fc-b0b2-a13714c8c56e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.310549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8da10d-2253-4308-9ca6-182bdf943edf admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.316569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.317303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.317616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.317909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] PATH_INFO: `/v3/projects/4b73f97b01b9426a86056ace687ab700/users/32177563d6014ca5adfce45d7ca13a0f/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.321293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.323279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.323690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.324131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.324792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.326523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.336186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b73f97b01b9426a86056ace687ab700, user_id=32177563d6014ca5adfce45d7ca13a0f, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.337227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:52.337227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:59:52.337227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:59:52.341622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 778/1598] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.341786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.341889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.356640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954075e-d070-4fe5-9556-66ced192b79b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.363552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.364580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.365035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.365564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.366439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.367693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 821/1599] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/4b73f97b01b9426a86056ace687ab700/users/32177563d6014ca5adfce45d7ca13a0f/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:52.368570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.369122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.404750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 779/1600] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/users => generated 296 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:52.406292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.419814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b2c5f8-cf86-4489-b7b1-d2ed27de7d10 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.450484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2410582-cb09-474e-b0dc-eb33e5b5dafc admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.453735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:52.458881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.459606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.460017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.460692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.461003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.462707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.474951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 822/1601] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.475899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.488988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.489387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d80dd0-adf6-45fb-b32e-9b85cc0cfccd admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.489604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.489891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.490165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.490817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.492501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.495383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.496173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.496493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.496840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] PATH_INFO: `/v3/projects/bd1b762323074c5fb8787a84faeacdbf/users/dd0e16d8533741e8bfa81e04152cae3a/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.505936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 780/1602] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.506885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.515674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd1b762323074c5fb8787a84faeacdbf, user_id=dd0e16d8533741e8bfa81e04152cae3a, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.518498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.518683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e1856-f44e-4587-a7f9-b00b0cea9457 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.523340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.524026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.524310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.524583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] PATH_INFO: `/v3/projects/eaddddfc3d754eee9390564296b7bd9b/users/1ab5ffa240dc4f06b33d0842aaf27392/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.531941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 823/1603] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/bd1b762323074c5fb8787a84faeacdbf/users/dd0e16d8533741e8bfa81e04152cae3a/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:52.533166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.539873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaddddfc3d754eee9390564296b7bd9b, user_id=1ab5ffa240dc4f06b33d0842aaf27392, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.543030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462632bf-7147-45ab-a4b5-cc36c486c80a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.556999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 781/1604] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/eaddddfc3d754eee9390564296b7bd9b/users/1ab5ffa240dc4f06b33d0842aaf27392/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:52.558226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.558510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.558775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.595620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] MFA Rules not processed for user `32177563d6014ca5adfce45d7ca13a0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:52.596788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:52.601104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:52.614707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e7db5-8592-4963-ab82-ad677a6c2d98 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.664144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 782/1605] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:52.665058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.673017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.673701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.673899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.674224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.674905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.676540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.687064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 824/1606] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.687862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.697881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6f512-fc29-4264-8203-deb737c79e89 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.704126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.704826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.705057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.705294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] PATH_INFO: `/v3/projects/9a2a3741defe4613bc01a88bce3c0bf7/users/b99bdb65eb5c472ab7711735f3d35a60/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.722340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a2a3741defe4613bc01a88bce3c0bf7, user_id=b99bdb65eb5c472ab7711735f3d35a60, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.725231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.740086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 825/1607] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/9a2a3741defe4613bc01a88bce3c0bf7/users/b99bdb65eb5c472ab7711735f3d35a60/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:52.740861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.749288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824965d-f1ac-490d-bf15-7b7e6d7568af None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.783070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:52.805512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5ced-f3ab-4c91-8bff-d74b2e742143 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:52.808935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.809549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.810004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.810132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:52.810404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.811279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.811640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.811836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.812052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] PATH_INFO: `/v3/projects/588cfe4bc032419f901b4989248d9a0f/users/88dab5446aff4214b3c15c7bb1d3b79c/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.813611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a696905-2d18-4f9a-90c0-e92c0dab8e6f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.824872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 783/1608] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:52.825980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f06-5994-4fd2-ad3a-d3861c9b89b5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.826284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f06-5994-4fd2-ad3a-d3861c9b89b5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.827344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84067f06-5994-4fd2-ad3a-d3861c9b89b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.830927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] RBAC: Authorizing `identity:create_grant(project_id=588cfe4bc032419f901b4989248d9a0f, user_id=88dab5446aff4214b3c15c7bb1d3b79c, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:52.833676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f4c15-59d6-4d4b-a8da-9d5aed0f554a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:52.848348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 826/1609] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/588cfe4bc032419f901b4989248d9a0f/users/88dab5446aff4214b3c15c7bb1d3b79c/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:52.849584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621b1e-ef90-4310-a748-1b2d53ce8a0c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.849879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621b1e-ef90-4310-a748-1b2d53ce8a0c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.850153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16621b1e-ef90-4310-a748-1b2d53ce8a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.862306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-84067f06-5994-4fd2-ad3a-d3861c9b89b5 None None] MFA Rules not processed for user `dd0e16d8533741e8bfa81e04152cae3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:52.863530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-84067f06-5994-4fd2-ad3a-d3861c9b89b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:52.880673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-16621b1e-ef90-4310-a748-1b2d53ce8a0c None None] MFA Rules not processed for user `1ab5ffa240dc4f06b33d0842aaf27392`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:52.881910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-16621b1e-ef90-4310-a748-1b2d53ce8a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:52.926165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:52.936277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 784/1610] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:52.937712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a0dd-ad53-45ac-9cbd-275a66c3d6bd None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.938097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a0dd-ad53-45ac-9cbd-275a66c3d6bd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.938464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13a0dd-ad53-45ac-9cbd-275a66c3d6bd None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.939992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 785/1611] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 24 09:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:52.940617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a0dd-ad53-45ac-9cbd-275a66c3d6bd None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:52.957605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 827/1612] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:52.960700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:52.961569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:52.961851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:52.982944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:52.984443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.015821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13a0dd-ad53-45ac-9cbd-275a66c3d6bd None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.020374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.021033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.021304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.021594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] PATH_INFO: `/v3/projects/a898c4356b99493197e2d5b0164c71fa/users/633f84ede14d49f19d9d4afacdcc80e5/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.035814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a898c4356b99493197e2d5b0164c71fa, user_id=633f84ede14d49f19d9d4afacdcc80e5, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.038476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2608afa2-eee0-4d15-a7ac-f1acb053c370 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.047529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 828/1613] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:53.048451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.052620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 786/1614] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:52 2022] PUT /identity/v3/projects/a898c4356b99493197e2d5b0164c71fa/users/633f84ede14d49f19d9d4afacdcc80e5/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:53.054145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.054435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.054709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.066786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d04ea1-b22f-4b03-96ad-b134b26a20f2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.075281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:53.077204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.121085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:53.125850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.126523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.126804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.127082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] PATH_INFO: `/v3/projects/ef79745c99e845fc993df1fbcdeaa71a/users/98b29975d9d2438aaf4a0eb85b266a2d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.141631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 787/1615] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:53.142871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79745c99e845fc993df1fbcdeaa71a, user_id=98b29975d9d2438aaf4a0eb85b266a2d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.144624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.150082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258217-40fb-4daf-84bc-41ea6dc38856 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.156146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d919d5-93ac-40f2-bd7b-fa04b1b2e268 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.161368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.161984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.162262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.162532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.163154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.165344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d847fc-dfb0-4ec8-993e-ed0346752511 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.166603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 829/1616] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:53 2022] PUT /identity/v3/projects/ef79745c99e845fc993df1fbcdeaa71a/users/98b29975d9d2438aaf4a0eb85b266a2d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:53.167943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90006a-c31b-48ec-8fc8-f7869c306363 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.168227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90006a-c31b-48ec-8fc8-f7869c306363 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.168487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90006a-c31b-48ec-8fc8-f7869c306363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.196832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 788/1617] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:53.196832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da331a4-9237-40c6-a3ff-2d75bb279096 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.196832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da331a4-9237-40c6-a3ff-2d75bb279096 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.196832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da331a4-9237-40c6-a3ff-2d75bb279096 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.197293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 789/1618] 10.222.0.78 () {54 vars in 962 bytes} [Wed Aug 24 09:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:53.197962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da331a4-9237-40c6-a3ff-2d75bb279096 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.212949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3d90006a-c31b-48ec-8fc8-f7869c306363 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:53.214562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3d90006a-c31b-48ec-8fc8-f7869c306363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.254396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da331a4-9237-40c6-a3ff-2d75bb279096 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.258040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.258637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.258918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.259195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.263644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:53.288009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 830/1619] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:53.289365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.289752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.290124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.318343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] MFA Rules not processed for user `633f84ede14d49f19d9d4afacdcc80e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:53.319595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.328458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.329397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.340491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 790/1620] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:53.341080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.349918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cdffa-85ab-4193-bf38-4ad76349eb78 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.353920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.354484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.354761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.355037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.365597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.367226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.378545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 791/1621] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:53.379463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.396018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e60f27-4895-4bbc-8424-00a0e9899427 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.401026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.401637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.402002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.402362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.404082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 831/1622] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:53.404231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.404339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.406099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8ab21-295d-4c2b-a2fd-3150a9dec61f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.415195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6553806e-9a71-45bd-9294-60643ac33f55 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.417879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 792/1623] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:53.419497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.419630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.419882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.422808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.423229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.423572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.423999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.424624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.426775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.442140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:53.443740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.454840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 793/1624] 10.222.0.78 () {58 vars in 1059 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:53.455738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.459496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 832/1625] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/users => generated 292 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:53.459496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.471306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f32d3-e9d3-4b4c-9686-d08a1a08d6d7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.473546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b1cce-e0af-414e-bbee-db1a39e7d2ae None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.476699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.477372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.477722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.478072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.478772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.480044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.480884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.481232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.481684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.481801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.514530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 833/1626] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/projects => generated 405 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:53.515674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.544293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:53.567664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.569523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.582470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 794/1627] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:53.583580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.596650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e520fd5-c26b-4258-aa7b-12724426064c service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.602260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.602875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.603262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.603434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.604948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.606270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a488ebc-585c-4c68-a027-5f5711a46237 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.606706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.613684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.614374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.615384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.615384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] PATH_INFO: `/v3/projects/ef79745c99e845fc993df1fbcdeaa71a/users/98b29975d9d2438aaf4a0eb85b266a2d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.617342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef79745c99e845fc993df1fbcdeaa71a, user_id=98b29975d9d2438aaf4a0eb85b266a2d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.618951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 795/1628] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:53.620585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.623900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.635237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 834/1629] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:53 2022] PUT /identity/v3/projects/ef79745c99e845fc993df1fbcdeaa71a/users/98b29975d9d2438aaf4a0eb85b266a2d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:53.645444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.691055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:53.737749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3bcec-0df1-4dca-9030-ca58c11f04c8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.739497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce506d45-b83a-4080-a67f-96a808b10427 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.746184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.747456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.747761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.749226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.750499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.753002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.788674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 835/1630] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/users => generated 309 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 09:59:53.790181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:53.809249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2de2a-886a-41d2-adba-91274994d960 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:53.820390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.821244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.821510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.821847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] PATH_INFO: `/v3/projects/752364a7b30143819edc54913b0b8759/users/6dff0704ef7a4d77848bb9fb7c45a00d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.845280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752364a7b30143819edc54913b0b8759, user_id=6dff0704ef7a4d77848bb9fb7c45a00d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.848958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046a69e1-eb99-4f9a-8c29-947d9ed65315 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.852049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:53.852764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.853118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.853975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.870086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 836/1631] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:53 2022] PUT /identity/v3/projects/752364a7b30143819edc54913b0b8759/users/6dff0704ef7a4d77848bb9fb7c45a00d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:53.871518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c74481-a522-4f8a-a373-0307b6c9ef56 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:53.871824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c74481-a522-4f8a-a373-0307b6c9ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:53.873133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c74481-a522-4f8a-a373-0307b6c9ef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:53.907474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f1c74481-a522-4f8a-a373-0307b6c9ef56 None None] MFA Rules not processed for user `98b29975d9d2438aaf4a0eb85b266a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:53.909706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f1c74481-a522-4f8a-a373-0307b6c9ef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:53.946222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:53.948599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:53.961758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 796/1632] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:53.975130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.002276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86a1ad-2bc2-4309-b8f4-593c476a55a3 service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.013648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:54.097022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 837/1633] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:54.098689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137dcb-8a3c-450d-9557-141009bd5213 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.098965 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137dcb-8a3c-450d-9557-141009bd5213 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.099242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94137dcb-8a3c-450d-9557-141009bd5213 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.101547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 838/1634] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 24 09:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:54.102452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137dcb-8a3c-450d-9557-141009bd5213 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.157153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.157877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.158071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.158347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.159001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.161225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.175001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 797/1635] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:54.176007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.239100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94137dcb-8a3c-450d-9557-141009bd5213 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.246272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.247028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.247410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.247821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.249654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.251371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96077f3-bfc5-4ffb-9c33-cd79643a454a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.256459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2ac0-e375-42ac-8416-23dc0e2fa3c0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.262331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.262964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.263308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.263595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.265271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 839/1636] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:54.266780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96088-81c6-4bc2-9b7b-84ac73a4aff0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.269966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96088-81c6-4bc2-9b7b-84ac73a4aff0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.269966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad96088-81c6-4bc2-9b7b-84ac73a4aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.300080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.305014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.322668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bad96088-81c6-4bc2-9b7b-84ac73a4aff0 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:54.325075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bad96088-81c6-4bc2-9b7b-84ac73a4aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:54.332261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 798/1637] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:54.334413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.341573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:54.359899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e05e9-6f9f-491f-b247-41a93db58a82 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.365570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.366692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.367433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.368171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] PATH_INFO: `/v3/projects/6ace5528f0f54ed09dff3255b7c7780f/users/d16db32232644e86acab569706f888b8/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.396896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ace5528f0f54ed09dff3255b7c7780f, user_id=d16db32232644e86acab569706f888b8, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.400487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.430670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 799/1638] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:54 2022] PUT /identity/v3/projects/6ace5528f0f54ed09dff3255b7c7780f/users/d16db32232644e86acab569706f888b8/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:54.431652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.442490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 840/1639] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:54.463836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0e0-b77e-4585-83c7-61106a91c67c None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.463836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0e0-b77e-4585-83c7-61106a91c67c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.463836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0e0-b77e-4585-83c7-61106a91c67c None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.463836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 841/1640] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 24 09:59:54 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 09:59:54.463836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4b0e0-b77e-4585-83c7-61106a91c67c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.547894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bb5a7a-cf7f-4528-bfd6-a3344de9cc87 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.549224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4b0e0-b77e-4585-83c7-61106a91c67c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.558056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.558754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.559199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.559452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.560819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.562616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.563066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610bbd1a-d13a-461a-bcfe-18c025dbad86 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.563313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.563574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.563878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] PATH_INFO: `/v3/projects/752364a7b30143819edc54913b0b8759/users/6dff0704ef7a4d77848bb9fb7c45a00d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.568585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=752364a7b30143819edc54913b0b8759, user_id=6dff0704ef7a4d77848bb9fb7c45a00d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.573101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.574842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 842/1641] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 09:59:54 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 09:59:54.576377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.576718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.577009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.592659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 800/1642] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:54 2022] PUT /identity/v3/projects/752364a7b30143819edc54913b0b8759/users/6dff0704ef7a4d77848bb9fb7c45a00d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:54.593859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.603689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:54.605184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:54.626793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:54.683058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562cb2d-45df-48d5-8b98-59a8f6d3a9b9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.689335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.691516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.692093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.692601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.708695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 843/1643] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:54.709924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.739447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b32ed77-c4b6-4a8a-9d9a-a87c7a79c81a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:54.743778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:54.744523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.744869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.745219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.814357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:54.831560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.835325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683bb9b-4b7b-4377-a4aa-15f884d51556 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.857292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:54.858978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:54.870902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 844/1644] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:54.872101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:54.882936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 801/1645] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:54.884434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:54.885093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:54.885093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:54.973861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] MFA Rules not processed for user `6dff0704ef7a4d77848bb9fb7c45a00d`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:54.975578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:55.024920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e598f22-da2c-4fb5-8afd-a2ed1e79e32e service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:55.039761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:55.040489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:55.041975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:55.042261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] PATH_INFO: `/v3/projects/6ace5528f0f54ed09dff3255b7c7780f/users/d16db32232644e86acab569706f888b8/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:55.049231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ace5528f0f54ed09dff3255b7c7780f, user_id=d16db32232644e86acab569706f888b8, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:55.051948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358df0a0-3a8b-4bb2-a217-f4681d4b057a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:55.089077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 845/1646] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 09:59:54 2022] PUT /identity/v3/projects/6ace5528f0f54ed09dff3255b7c7780f/users/d16db32232644e86acab569706f888b8/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:55.100505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:55.102200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:55.102665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:55.142117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 802/1647] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:55.168539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] MFA Rules not processed for user `d16db32232644e86acab569706f888b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:55.171283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:55.293717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 846/1648] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:55 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 09:59:55.363964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:55.377327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca91f47-5ebc-4750-8e84-a3efc157fd11 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:55.424132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:55.473387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:55.485750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:55.486450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:55.486737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:55.487010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:55.590941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb80dc-5137-4e3a-89cb-1e9bc2f2f300 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:55.601639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:55.602351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:55.602635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:55.602913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:55.624963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:55.654331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:55.658823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:55.685644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 803/1649] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:55.734607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:55.744645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054c06-d85d-4a0c-b133-ea0e0d4316a5 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:55.753051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:55.753597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:55.753899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:55.754146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:55.763877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:55.767653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:55.776969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:55.778436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:55.794605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 847/1650] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:55.817242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 804/1651] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:55 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:57.101891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:57.159882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:57.285893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b22358-3e99-44e5-b5f7-fef47eed9415 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:57.298019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87429fef-2d89-4037-bc54-8f1aa2797c67 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:57.302164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:57.303283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:57.303630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:57.305007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] PATH_INFO: `/v3/users/633f84ede14d49f19d9d4afacdcc80e5` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:57.307366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=633f84ede14d49f19d9d4afacdcc80e5)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:57.311914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:57.325278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:57.326336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:57.328239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:57.328846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] PATH_INFO: `/v3/users/1ab5ffa240dc4f06b33d0842aaf27392` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:57.330900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ab5ffa240dc4f06b33d0842aaf27392)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:57.332230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: WARNING py.warnings [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:57.332230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: warnings.warn(msg) Aug 24 09:59:57.332230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: Aug 24 09:59:57.333985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:57.450653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '633f84ede14d49f19d9d4afacdcc80e5'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.451351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '633f84ede14d49f19d9d4afacdcc80e5'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.460684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ab5ffa240dc4f06b33d0842aaf27392'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.462728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ab5ffa240dc4f06b33d0842aaf27392'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.492202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ab5ffa240dc4f06b33d0842aaf27392'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.492468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '633f84ede14d49f19d9d4afacdcc80e5'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.503755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '633f84ede14d49f19d9d4afacdcc80e5'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.505907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:57.522791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7f4e4714-d8bf-4cc1-8b9d-2bf4cbe670c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '633f84ede14d49f19d9d4afacdcc80e5'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.536954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 805/1652] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:57 2022] DELETE /identity/v3/users/633f84ede14d49f19d9d4afacdcc80e5 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:57.538224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ab5ffa240dc4f06b33d0842aaf27392'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.539085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:57.539483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:57.539885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:57.566900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:57.569283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:57.582988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f9f42f4c-2fcd-4c30-931a-93d0904277a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ab5ffa240dc4f06b33d0842aaf27392'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:57.599113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 848/1653] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:57 2022] DELETE /identity/v3/users/1ab5ffa240dc4f06b33d0842aaf27392 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:57.600840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:57.601304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:57.601788 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:57.669399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:57.671116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:57.732659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:57.772859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 806/1654] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:57.798165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:57.798165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:57.854146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 849/1655] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:57.864683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:57.866140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:57.916643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffffc15-02bb-4636-ba7e-b169949d3888 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:57.987224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c551b640-3e21-4a37-9d74-618a9824459e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:58.003296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:58.004161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:58.004545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:58.005075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:58.024286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:58.026130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:58.054547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:58.055638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:58.055638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:58.055889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:58.072638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 850/1656] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 09:59:58.076045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:58.076601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:58.078374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:58.080114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03508d-b51f-4d20-bc70-ff2ae2c0b680 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:58.102629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0de77-a6ff-49cd-b2ce-3feef55a59de service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:58.110722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:58.111393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:58.111679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:58.112062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] PATH_INFO: `/v3/users/dd0e16d8533741e8bfa81e04152cae3a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:58.113256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0e16d8533741e8bfa81e04152cae3a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:58.115516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:58.122601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 807/1657] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:58.188365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0e16d8533741e8bfa81e04152cae3a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:58.190858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0e16d8533741e8bfa81e04152cae3a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:58.208923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0e16d8533741e8bfa81e04152cae3a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:58.233063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0e16d8533741e8bfa81e04152cae3a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:58.251027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0e16d8533741e8bfa81e04152cae3a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:58.259939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 851/1658] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:58 2022] DELETE /identity/v3/users/dd0e16d8533741e8bfa81e04152cae3a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:58.263657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:58.264128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:58.264259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:58.292515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:58.293757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:58.410278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:58.470373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 808/1659] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:58.588654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:58.590830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:58.714823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:58.765843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4596f25-0dd2-4242-b1f0-64206046234c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:58.779182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:58.780527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:58.780704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:58.781077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:58.814646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:58.819177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:58.881287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 852/1660] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 09:59:59.028951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:59.031410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.059926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6dbdc9-0355-46dc-9f2f-092bef649b1e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:59.085369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:59.085962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.086269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.093598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] PATH_INFO: `/v3/users/32177563d6014ca5adfce45d7ca13a0f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.097287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=32177563d6014ca5adfce45d7ca13a0f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:59.104116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:59.207218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:59.208907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.253935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a8934-d16f-40f8-8d04-85014180f5e5 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:59.266620 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32177563d6014ca5adfce45d7ca13a0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.269272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32177563d6014ca5adfce45d7ca13a0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.279260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:59.280707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.281137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.281520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] PATH_INFO: `/v3/projects/a898c4356b99493197e2d5b0164c71fa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.284003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a898c4356b99493197e2d5b0164c71fa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:59.287720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:59.300654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32177563d6014ca5adfce45d7ca13a0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.318817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32177563d6014ca5adfce45d7ca13a0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.340736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ee45cdd8-931e-4f17-8cf4-b7afd2857ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32177563d6014ca5adfce45d7ca13a0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.356364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 809/1661] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:59 2022] DELETE /identity/v3/users/32177563d6014ca5adfce45d7ca13a0f => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:59.362389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.362848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.364353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.440146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:59.448798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a898c4356b99493197e2d5b0164c71fa', 'request_id': 'req-c6b6e809-8405-4d5a-ae2c-24f13576b380'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.454364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.469592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a898c4356b99493197e2d5b0164c71fa', 'request_id': 'req-c6b6e809-8405-4d5a-ae2c-24f13576b380'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.477181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a898c4356b99493197e2d5b0164c71fa', 'request_id': 'req-c6b6e809-8405-4d5a-ae2c-24f13576b380'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.511134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] The token cache is being invalidate because project a898c4356b99493197e2d5b0164c71fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:59:59.511692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.513615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.519434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 853/1662] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 09:59:59 2022] DELETE /identity/v3/projects/a898c4356b99493197e2d5b0164c71fa => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:59.523044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:59.524715 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.622568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 810/1663] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 09:59:59.634369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:59.636521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.660118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade854d-448a-4a2d-83b3-e8fc559115d7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:59.679198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:59.680175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.680828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.681326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] PATH_INFO: `/v3/projects/eaddddfc3d754eee9390564296b7bd9b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.685650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaddddfc3d754eee9390564296b7bd9b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:59.687845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: WARNING py.warnings [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 09:59:59.687845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: warnings.warn(msg) Aug 24 09:59:59.687845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: Aug 24 09:59:59.689527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:59.698701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6e809-8405-4d5a-ae2c-24f13576b380 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:59.714819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 09:59:59.715738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.716416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.716932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] PATH_INFO: `/v3/users/d16db32232644e86acab569706f888b8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.719559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d16db32232644e86acab569706f888b8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 09:59:59.721942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 09:59:59.797244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd16db32232644e86acab569706f888b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.798068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd16db32232644e86acab569706f888b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.812448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd16db32232644e86acab569706f888b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.830653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:59.834209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaddddfc3d754eee9390564296b7bd9b', 'request_id': 'req-f6b98adf-67af-448d-af8f-f547c122ace6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.849649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd16db32232644e86acab569706f888b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.851927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaddddfc3d754eee9390564296b7bd9b', 'request_id': 'req-f6b98adf-67af-448d-af8f-f547c122ace6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.854135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 09:59:59.862072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd16db32232644e86acab569706f888b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.876303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 854/1664] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 09:59:59 2022] DELETE /identity/v3/users/d16db32232644e86acab569706f888b8 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 09:59:59.877513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 09:59:59.879472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.888522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaddddfc3d754eee9390564296b7bd9b', 'request_id': 'req-f6b98adf-67af-448d-af8f-f547c122ace6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.916048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] The token cache is being invalidate because project eaddddfc3d754eee9390564296b7bd9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 09:59:59.916341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.920198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f6b98adf-67af-448d-af8f-f547c122ace6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 09:59:59.923158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 811/1665] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 09:59:59 2022] DELETE /identity/v3/projects/eaddddfc3d754eee9390564296b7bd9b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 09:59:59.928386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d03bd1-54a0-444a-bb8f-d383122f0bac None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 09:59:59.928554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d03bd1-54a0-444a-bb8f-d383122f0bac None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 09:59:59.929027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d03bd1-54a0-444a-bb8f-d383122f0bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 09:59:59.972358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-38d03bd1-54a0-444a-bb8f-d383122f0bac None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 09:59:59.972938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-38d03bd1-54a0-444a-bb8f-d383122f0bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 09:59:59.992401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5198690-4877-46d2-8bed-43afcb46ccef admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 09:59:59.999821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.001138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.001568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.001859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.005573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:00.124803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.127184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.130370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 812/1666] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 09:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:00.136476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.140193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.140529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.149288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 855/1667] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 09:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:00.179274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.181274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:00.192515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:00.194021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:00.214662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947d507-a4a8-4eab-82ce-1275bb33f17d service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.218818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 813/1668] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:00.225590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.227249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.228620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.228910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.229264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] PATH_INFO: `/v3/projects/bd1b762323074c5fb8787a84faeacdbf` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.230780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd1b762323074c5fb8787a84faeacdbf)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.235385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.248866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096fb2-36a8-41cd-b5ed-79f7c6aba2c3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.261899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.262529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.262811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.263087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.266370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:00.266683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.269944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.299523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 814/1669] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:00.301066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.321702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907037d-8f35-40d0-ab89-43e57dfbc7d7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.338243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd1b762323074c5fb8787a84faeacdbf', 'request_id': 'req-121dfcc0-8568-4c61-b890-5ece21b5350c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.350429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd1b762323074c5fb8787a84faeacdbf', 'request_id': 'req-121dfcc0-8568-4c61-b890-5ece21b5350c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.363983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd1b762323074c5fb8787a84faeacdbf', 'request_id': 'req-121dfcc0-8568-4c61-b890-5ece21b5350c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.379510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] The token cache is being invalidate because project bd1b762323074c5fb8787a84faeacdbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:00.379834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.380553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.385473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 856/1670] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:00 2022] DELETE /identity/v3/projects/bd1b762323074c5fb8787a84faeacdbf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:00.386627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.390179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:00.429488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:00.462283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.463085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.463198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.463471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.464143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.466098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.553704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 815/1671] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:00 2022] POST /identity/v3/projects => generated 353 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:00.560920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.564033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:00.571534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dfcc0-8568-4c61-b890-5ece21b5350c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.715980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.716634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.716943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.717196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] PATH_INFO: `/v3/users/6dff0704ef7a4d77848bb9fb7c45a00d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.720660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dff0704ef7a4d77848bb9fb7c45a00d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.722655 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.729063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:00.760333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f13f9-f7f3-49ef-aa20-f6c626e5fb47 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.774701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.775361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.775651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.775963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.776814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.778970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.835948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 816/1672] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:00 2022] POST /identity/v3/users => generated 277 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:00.837385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.843364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dff0704ef7a4d77848bb9fb7c45a00d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.844097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dff0704ef7a4d77848bb9fb7c45a00d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.855253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dff0704ef7a4d77848bb9fb7c45a00d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.869064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8678e40-7fcd-40c0-9d91-22b9b750005c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:00.909031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dff0704ef7a4d77848bb9fb7c45a00d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.910021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:00.911099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:00.911526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:00.912007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:00.913908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:00.918684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dff0704ef7a4d77848bb9fb7c45a00d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:00.929487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:00.933864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 857/1673] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:00 2022] DELETE /identity/v3/users/6dff0704ef7a4d77848bb9fb7c45a00d => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:00.958936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 817/1674] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:00.963274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:00.965152 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:01.058375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:01.124401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8148aa1d-e917-480f-a015-d8c904c90864 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:01.131546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:01.132513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:01.132962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:01.133445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:01.134248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:01.136282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf70bf-5c70-43cc-b619-0d76e22ac564 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:01.159495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 858/1675] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:01.161524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:01.198072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fc834c-eb08-4af6-90aa-0cd92b78d903 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:01.219545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:01.222670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:01.222976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:01.224877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] PATH_INFO: `/v3/projects/2cc5dd59014b49c39d23c1b613c818e2/users/11a0c98f7d4749228eefb12b1fa3c425/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:01.295186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc5dd59014b49c39d23c1b613c818e2, user_id=11a0c98f7d4749228eefb12b1fa3c425, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:01.304734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:01.344044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 818/1676] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:01 2022] PUT /identity/v3/projects/2cc5dd59014b49c39d23c1b613c818e2/users/11a0c98f7d4749228eefb12b1fa3c425/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:01.354834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:01.355283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:01.355480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:01.427046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] MFA Rules not processed for user `11a0c98f7d4749228eefb12b1fa3c425`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:01.428348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:01.634996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:01.636971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:01.648020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 859/1677] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:01.650599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:01.925809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b0835-b91a-4ab7-ad59-f61ca3baff2b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:01.947542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:01.948874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:01.949225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:01.949997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] PATH_INFO: `/v3/projects/6ace5528f0f54ed09dff3255b7c7780f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:01.956330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ace5528f0f54ed09dff3255b7c7780f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:01.965395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:02.031746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671acd9-5107-48c8-8453-e2d39987cabd None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:02.044972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:02.045530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.047178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.048482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.050275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:02.055016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:02.115024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ace5528f0f54ed09dff3255b7c7780f', 'request_id': 'req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.145645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ace5528f0f54ed09dff3255b7c7780f', 'request_id': 'req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.149485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 860/1678] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:01 2022] POST /identity/v3/projects => generated 355 bytes in 501 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:02.151063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:02.153712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:02.162390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ace5528f0f54ed09dff3255b7c7780f', 'request_id': 'req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.192015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] The token cache is being invalidate because project 6ace5528f0f54ed09dff3255b7c7780f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:02.194475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.196839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.200166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 819/1679] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:01 2022] DELETE /identity/v3/projects/6ace5528f0f54ed09dff3255b7c7780f => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:02.201073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:02.205207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:02.236388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:02.291710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a73868-735c-416f-8506-c18c18de98b1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:02.307471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:02.308578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:02.310329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.312128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.312608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] PATH_INFO: `/v3/projects/4b73f97b01b9426a86056ace687ab700` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.319842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b73f97b01b9426a86056ace687ab700)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:02.326557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:02.390339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fe9a5-2890-4bb1-aa45-e1e5d709b7e2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:02.411454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:02.412135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.412420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.412794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] PATH_INFO: `/v3/projects/752364a7b30143819edc54913b0b8759` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.416471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=752364a7b30143819edc54913b0b8759)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:02.418611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:02.499093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b73f97b01b9426a86056ace687ab700', 'request_id': 'req-7a2f7580-a465-4125-a28b-838ccb90aec9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.517995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b73f97b01b9426a86056ace687ab700', 'request_id': 'req-7a2f7580-a465-4125-a28b-838ccb90aec9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.529814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b73f97b01b9426a86056ace687ab700', 'request_id': 'req-7a2f7580-a465-4125-a28b-838ccb90aec9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.543965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '752364a7b30143819edc54913b0b8759', 'request_id': 'req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.553387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] The token cache is being invalidate because project 4b73f97b01b9426a86056ace687ab700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:02.555004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.555483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.558472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 861/1680] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:02 2022] DELETE /identity/v3/projects/4b73f97b01b9426a86056ace687ab700 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:02.559455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:02.561008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:02.568306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '752364a7b30143819edc54913b0b8759', 'request_id': 'req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.581419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '752364a7b30143819edc54913b0b8759', 'request_id': 'req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.619244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] The token cache is being invalidate because project 752364a7b30143819edc54913b0b8759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:02.620756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.624135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-13e1a904-cd4a-4b2c-a23d-3de0d91d5a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:02.628388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 820/1681] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:02 2022] DELETE /identity/v3/projects/752364a7b30143819edc54913b0b8759 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:02.633988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69cb2b-5900-4a2c-b7a5-f74096e32543 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.634305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69cb2b-5900-4a2c-b7a5-f74096e32543 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.635045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f69cb2b-5900-4a2c-b7a5-f74096e32543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.703254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1f69cb2b-5900-4a2c-b7a5-f74096e32543 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:02.705909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1f69cb2b-5900-4a2c-b7a5-f74096e32543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:02.786552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f7580-a465-4125-a28b-838ccb90aec9 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:02.890640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 821/1682] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:02.890640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98607b5f-4253-4b41-991e-d77208d62beb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.890640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98607b5f-4253-4b41-991e-d77208d62beb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.890640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98607b5f-4253-4b41-991e-d77208d62beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.932615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-98607b5f-4253-4b41-991e-d77208d62beb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:02.934720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-98607b5f-4253-4b41-991e-d77208d62beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:02.977849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:02.979068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.979563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.980096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:02.980932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:02.985725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aeafb2-dc5a-4ac9-bd2f-6c27c3355085 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:02.986485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 822/1683] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:02.988651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0702-82d0-4c3c-893c-058ef33acbdd None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:02.989122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0702-82d0-4c3c-893c-058ef33acbdd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:02.989573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0702-82d0-4c3c-893c-058ef33acbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.039255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 862/1684] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:02 2022] POST /identity/v3/users => generated 284 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:03.042521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.042877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.051119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.086346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0f1d0702-82d0-4c3c-893c-058ef33acbdd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:03.088414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0f1d0702-82d0-4c3c-893c-058ef33acbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:03.090335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:03.092393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:03.123345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 823/1685] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:03.124936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 863/1686] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:03.130497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.131109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.131394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.131667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.137878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:03.165248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7a299b-2e64-43c5-9175-e68759240a49 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.177765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:03.178939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.180089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.180674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.180980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.182830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.183859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:03.187624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3052ed4a-3620-4fd7-b3ae-245bb5779022 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.205245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 824/1687] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:03.211964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.216918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 864/1688] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:03.222889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.223585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.225427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.244037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ffab-66d9-4778-b33c-f7663ffa11d9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.250012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:03.263314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.266938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.267503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.268052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.270472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:03.271211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.274163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.290441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:03.294349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:03.295373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 825/1689] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:03.297489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.313104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 865/1690] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:03.314147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.318886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e5c45-c75e-4d2e-85a5-f5e769e21143 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.332188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.333232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.333884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe5b6a-6d98-450d-8e65-e5693be00931 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.334194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.334716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.335963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:03.348793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.349403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.350771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.351063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.351345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] PATH_INFO: `/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34/users/e2d6ed8dcc4d469ea4b59abd5a8569cc/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.351834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.386081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aedf6a9259f4f05a93e891e4aca1f34, user_id=e2d6ed8dcc4d469ea4b59abd5a8569cc, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.387049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 826/1691] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:03.388123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.388867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.403723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02c52cd-d71c-44cf-8eaa-d842da1291d0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.410447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.411189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.411471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.411744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.413073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.415634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.433369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 866/1692] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:03 2022] PUT /identity/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34/users/e2d6ed8dcc4d469ea4b59abd5a8569cc/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:03.434561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.463691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:03.479524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 827/1693] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/projects => generated 367 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:03.480929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.616987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb724f6-b102-4355-ad66-15eb141003a3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.658079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05caf0b-2911-4575-bafb-7acb39b6ff23 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.676523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.677165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.677451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.677728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.678374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.681808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.745162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 828/1694] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:03.745162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.754346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.754929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.755226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.755492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.757160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:03.757474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.763365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.781849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 867/1695] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:03.782884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.802214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d602d-3f87-44e5-a349-29016145a501 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.877438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.877986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.878273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.878643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.879179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.882409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.890587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f6a0b-fa68-4717-8872-ba9d294648ef admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.902809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.903642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.903868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.904207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.905600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.907988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.919128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 868/1696] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/users => generated 284 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:03.920087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.931400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:03.948309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c929c-57a8-4693-b003-7d19c13a62be admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.950428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 829/1697] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:03.952960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:03.963109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.963491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.963815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.964129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.964822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.966785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:03.977661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6acf8cb-2409-40d2-a01f-67a31daea621 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:03.984776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:03.993112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:03.993665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:03.994211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:03.995252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:03.997782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.003362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 869/1698] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/users => generated 287 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:04.004345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.023667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddd967-c007-4297-aa3f-70ee273b8f1b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.030763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.033853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.036372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.036372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.036372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.037528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.049565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 830/1699] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:03 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:04.050999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.057419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 870/1700] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:04.058316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.088293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65471fe-2f85-4af5-be3e-4766800e4a76 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.201864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:04.213823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc9a6d-acaf-40d7-a12d-dd0e452234de admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.233450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.234938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.235253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.235706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] PATH_INFO: `/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34/users/e2d6ed8dcc4d469ea4b59abd5a8569cc/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.252892 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aedf6a9259f4f05a93e891e4aca1f34, user_id=e2d6ed8dcc4d469ea4b59abd5a8569cc, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.255677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.263830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.264472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.265704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.266003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.266805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.279677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.303571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 871/1702] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:04.305069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.310305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 831/1702] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:04 2022] PUT /identity/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34/users/e2d6ed8dcc4d469ea4b59abd5a8569cc/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:04.311410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.322171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569bb9c2-535d-4b29-b325-0ec669b7812f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.435977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.437530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.438180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.438469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.439276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.442941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.464346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd13af33-8a51-4e39-b5ca-af15e7c0d894 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.475593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.476946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.476946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.477165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.477663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.479278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76bac7c-07ed-4d0b-83a9-759c4ebcc027 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.486216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 872/1703] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:04 2022] POST /identity/v3/users => generated 282 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:04.488612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.520250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d314c93-ecdc-4db4-8739-c384f8985018 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.522079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 832/1704] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:04.523823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.524105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.524382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.532969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.533739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.534034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.534310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.537479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.540164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd01de00-b14e-4018-b7e3-bb7a9d1add31 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.574256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 873/1705] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:04.575776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb67b-7819-4159-9cde-fcf150cbe7f9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.576727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb67b-7819-4159-9cde-fcf150cbe7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.576727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9deb67b-7819-4159-9cde-fcf150cbe7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.594278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] MFA Rules not processed for user `e2d6ed8dcc4d469ea4b59abd5a8569cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:04.596931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:04.617155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b9deb67b-7819-4159-9cde-fcf150cbe7f9 None None] MFA Rules not processed for user `b99bdb65eb5c472ab7711735f3d35a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:04.618771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b9deb67b-7819-4159-9cde-fcf150cbe7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:04.685506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:04.703781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 833/1706] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:04.704783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.724184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 874/1707] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:04.724525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31270911-34cd-45c1-ab5c-d398a31b4db0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.726524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.728012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.728300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.744115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.745615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.745615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.745865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.746518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.748199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.761985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] MFA Rules not processed for user `88dab5446aff4214b3c15c7bb1d3b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:04.763255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:04.766561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 834/1708] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:04.767944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.784400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59829266-dcdb-4356-b9eb-424c50a550a2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.789730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.790401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.791236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.791516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] PATH_INFO: `/v3/projects/5cdcf98f95004bc081ad3ebd73124084/users/d05eb809bed54d34be15c63eaf56dd99/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.813409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cdcf98f95004bc081ad3ebd73124084, user_id=d05eb809bed54d34be15c63eaf56dd99, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:04.817039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:04.839558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 875/1709] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:04.840602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.846776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 835/1710] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:04 2022] PUT /identity/v3/projects/5cdcf98f95004bc081ad3ebd73124084/users/d05eb809bed54d34be15c63eaf56dd99/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:04.847888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:04.869652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:04.891518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd51daa-88d1-4c36-ae0e-30e6a692fb51 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.934625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:04.962576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb44c0f-c0be-4f2a-b199-c8ec1254e4bf admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:04.970519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.971531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.972173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.972683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:04.979875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:04.980560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:04.980847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:04.981123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] PATH_INFO: `/v3/projects/36ae13d7131b4bdfbef1cb92a750dda9/users/7e9e0768a9894388a01d37bb2aa340aa/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.004321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36ae13d7131b4bdfbef1cb92a750dda9, user_id=7e9e0768a9894388a01d37bb2aa340aa, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.008346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.027364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 876/1711] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:04 2022] PUT /identity/v3/projects/36ae13d7131b4bdfbef1cb92a750dda9/users/7e9e0768a9894388a01d37bb2aa340aa/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:05.036790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.038783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.072517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.074214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.089422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 836/1712] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:04 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:05.090210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.102469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed29eaa-0023-4eb8-95cb-41ef2725eedd service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.127214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:05.171496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff5e8f2-7fd1-4337-8884-646507758191 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.179407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:05.180659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.181108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.181441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.207042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:05.207815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.208195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.209282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.210086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.212667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef782e4-fb0f-440c-aa6c-4de5dd6cf6b4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.227737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 837/1713] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:05.229293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbe5c-962a-4a12-84ec-f901b75163ee None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.229961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbe5c-962a-4a12-84ec-f901b75163ee None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.230157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbe5c-962a-4a12-84ec-f901b75163ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.260694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-772fbe5c-962a-4a12-84ec-f901b75163ee None None] MFA Rules not processed for user `88dab5446aff4214b3c15c7bb1d3b79c`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:05.262844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-772fbe5c-962a-4a12-84ec-f901b75163ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.271490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.274385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade7474-48e7-416e-9c44-331f4614eba8 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.287540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 877/1714] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:05.298864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.299157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.299443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.331941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] MFA Rules not processed for user `d05eb809bed54d34be15c63eaf56dd99`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:05.333401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.336704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 838/1715] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:05.338230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.338764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.339054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.386436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] MFA Rules not processed for user `7e9e0768a9894388a01d37bb2aa340aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:05.388254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.419838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:05.508961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 878/1716] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:05.510145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.612256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 839/1717] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:05.613253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.615292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2a6d19-ceaf-4b6e-b499-296d3027d727 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.630670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c46a0c-f3ab-45bf-b124-ec8b073bf677 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.633909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:05.636628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.636752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:05.636949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.637201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.637630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.637990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.638415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] PATH_INFO: `/v3/projects/3e9c6d1cab3447f6b1a8aad4893ef281/users/abaa343237d34a3099880f315a699125/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:05.648544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.661670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e9c6d1cab3447f6b1a8aad4893ef281, user_id=abaa343237d34a3099880f315a699125, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.665077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.707044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:05.709040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 840/1718] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:05 2022] PUT /identity/v3/projects/3e9c6d1cab3447f6b1a8aad4893ef281/users/abaa343237d34a3099880f315a699125/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:05.711199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.716045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:05.788528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:05.790376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:05.802285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 879/1719] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:05.803293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:05.855822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d411e-7381-4a43-8f82-51b8bd2dd504 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.876564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349a52e-a7d2-4a51-98ef-a7305cbf9bb0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:05.896629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:05.898944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:05.899304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:05.899659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.005773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.008813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.012219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.012590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.013711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.017908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250c25-3bbc-46b3-8155-d97369f18a88 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.071636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.073426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.097582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 841/1720] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:06.100901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.129284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc759607-1971-4d77-a24b-547c57a9192b service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.141420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.142574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.143378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.143751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.144640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.157012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.157391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 880/1721] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:05 2022] POST /identity/v3/projects => generated 369 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.161761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.162070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.162349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.228785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 842/1722] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.229811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.262776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] MFA Rules not processed for user `abaa343237d34a3099880f315a699125`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:06.266973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:06.276237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:06.391309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978c276-0004-45f7-a62f-4c198ec3b9a6 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.409821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.410724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.411149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.412184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.413128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.416734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.490634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 881/1723] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:06.494731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.500153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 843/1724] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.500153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.521446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632e7ae9-b785-484e-b86d-02199e04395f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.532010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b2812-b34c-4800-b996-9d08baaa2d82 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.536121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.539860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.539860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.539860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.539860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.543589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.544131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.544975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.545257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.545528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.546769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.554162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.604252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 882/1725] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.605304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.619200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 844/1726] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/projects => generated 359 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.620245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.659357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503a4de-3103-48d0-8b24-68a06bbb2ce6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.771205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b24365-c704-41a3-9298-a4235a18d301 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.777910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:06.784360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.785098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.785426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.785767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.786520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.788533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.811063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 845/1727] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:06.816129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.839351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.840162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.840636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.840983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.841829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.844705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5a3f2-c37c-4084-aaff-14a780da6a00 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.845754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.859248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 883/1728] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:06.860641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.862443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.863166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.863524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.863910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.864640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.866665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.878305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb2328-3c8a-4c85-abc2-68e2df52126b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.884342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:06.885373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:06.885852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:06.886378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] PATH_INFO: `/v3/projects/abba5d090bea476d93d0f554ad37b524/users/82395f2e0c5c417ea72f722e290c8bac/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:06.904791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=abba5d090bea476d93d0f554ad37b524, user_id=82395f2e0c5c417ea72f722e290c8bac, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:06.913724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:06.919865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 846/1729] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:06 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:06.920455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:06.943132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 884/1730] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:06 2022] PUT /identity/v3/projects/abba5d090bea476d93d0f554ad37b524/users/82395f2e0c5c417ea72f722e290c8bac/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:06.947302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3cca1-265d-4085-bda4-f45c0e9010b9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:06.948130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.129068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2a999-50c7-4db7-acaa-42e267deea9a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.141070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.141906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.142720 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.143254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.144437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.146265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.158622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:07.160853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 885/1731] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:06 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:07.161948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.173284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.174787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.175247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.175675 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] PATH_INFO: `/v3/projects/d7167a36020948898c747c672aee1aad/users/7cb40b416d7546f187fc5ecea6a3eff2/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.176598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300977c-b9ca-46ca-8087-afbc7ff5843e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.183977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.184777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.185087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.185360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.186063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.188651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7608a-2957-4af7-8494-5884e5f792fa admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.210310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 886/1732] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:07.217547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df12b-4ec7-4d27-ad42-75e590c54921 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.218019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df12b-4ec7-4d27-ad42-75e590c54921 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.218735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547df12b-4ec7-4d27-ad42-75e590c54921 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.227741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 887/1733] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 24 10:00:07 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:00:07.228871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df12b-4ec7-4d27-ad42-75e590c54921 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.233882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7167a36020948898c747c672aee1aad, user_id=7cb40b416d7546f187fc5ecea6a3eff2, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.237194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.267774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 847/1734] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:06 2022] PUT /identity/v3/projects/d7167a36020948898c747c672aee1aad/users/7cb40b416d7546f187fc5ecea6a3eff2/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:07.268630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.273208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547df12b-4ec7-4d27-ad42-75e590c54921 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.422071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.423034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.423448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.423752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] PATH_INFO: `/v3/projects/acf6cb08467f49f399a45342c4fe1260/users/9e5f6f21fcbe4aceb97069348c583c62/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.469183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf6cb08467f49f399a45342c4fe1260, user_id=9e5f6f21fcbe4aceb97069348c583c62, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.472107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eca539-a776-4cd2-ba9a-ea96e52ae606 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.478862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9376cd-50f6-4a17-8a6c-84654b50eece admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.497672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.500748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.502570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.504028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] PATH_INFO: `/v3/projects/abba5d090bea476d93d0f554ad37b524/users/82395f2e0c5c417ea72f722e290c8bac/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.511565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abba5d090bea476d93d0f554ad37b524, user_id=82395f2e0c5c417ea72f722e290c8bac, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.518543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 888/1735] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:07 2022] PUT /identity/v3/projects/acf6cb08467f49f399a45342c4fe1260/users/9e5f6f21fcbe4aceb97069348c583c62/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:07.522076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.525191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.525191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.525475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.556246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 848/1736] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:07 2022] PUT /identity/v3/projects/abba5d090bea476d93d0f554ad37b524/users/82395f2e0c5c417ea72f722e290c8bac/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:07.566360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.568501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:07.639142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:07.643626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:07.782816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 889/1737] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:07.784580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.795082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e055e-1ea9-4a63-998a-b95efea6e4b5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.814687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.816474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.816766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.817057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.821737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.822647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f471-6574-4b4b-ae7d-2950845a9390 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.825662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea457d-f61d-46fe-8f19-7cdfefd81e64 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.832606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.833662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.834154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.834900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.836262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.839164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.847576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 849/1738] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:07.853074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838e70-d973-49a6-8d8e-3f7851199a0b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.853402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838e70-d973-49a6-8d8e-3f7851199a0b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.853665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df838e70-d973-49a6-8d8e-3f7851199a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.876312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 890/1739] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:07.880222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:07.886224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:07.927921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d385895-78f5-40f0-b3d1-f20350dad6af admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:07.935094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:07.936228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:07.937182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:07.937824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:07.948518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-df838e70-d973-49a6-8d8e-3f7851199a0b None None] MFA Rules not processed for user `82395f2e0c5c417ea72f722e290c8bac`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:07.951609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-df838e70-d973-49a6-8d8e-3f7851199a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:07.954269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:07.957718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:07.988673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 891/1740] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:07.989859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:08.014408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ee949-8867-46eb-ac3f-b173f550dbc1 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:08.029776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:08.031139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.032608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.033107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] PATH_INFO: `/v3/projects/d7167a36020948898c747c672aee1aad/users/7cb40b416d7546f187fc5ecea6a3eff2/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.040293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7167a36020948898c747c672aee1aad, user_id=7cb40b416d7546f187fc5ecea6a3eff2, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:08.048079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:08.081811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:08.098795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 892/1741] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:07 2022] PUT /identity/v3/projects/d7167a36020948898c747c672aee1aad/users/7cb40b416d7546f187fc5ecea6a3eff2/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:08.098795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:08.103532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 850/1742] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:08.105151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.105568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.105948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.143607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] MFA Rules not processed for user `7cb40b416d7546f187fc5ecea6a3eff2`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:08.145032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:08.271709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 851/1743] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:08.276171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:08.279502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:08.294077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882ef05-4048-442b-8a77-2e046d9c5d70 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:08.308431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:08.310972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.311482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.312639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] PATH_INFO: `/v3/projects/acf6cb08467f49f399a45342c4fe1260/users/9e5f6f21fcbe4aceb97069348c583c62/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.317722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=acf6cb08467f49f399a45342c4fe1260, user_id=9e5f6f21fcbe4aceb97069348c583c62, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:08.323109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:08.349486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 893/1744] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:08 2022] PUT /identity/v3/projects/acf6cb08467f49f399a45342c4fe1260/users/9e5f6f21fcbe4aceb97069348c583c62/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:08.351943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:08.422637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0180d-78fe-4504-aea6-6469c9261edf None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:08.481522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdf711-222b-44b5-bf39-37b1cfe09bbf admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:08.488664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:08.489384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.489742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.490090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.513395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:08.545792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:08.546664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.547009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.547230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.628896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:08.633732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd34cfc-163c-4c96-9b41-b7069d55ffeb service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:08.655091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:08.678515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:08.680313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:08.681110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 894/1745] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:08.682424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8da9-8c2a-4323-8cdd-79ec659721cb None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:08.682718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8da9-8c2a-4323-8cdd-79ec659721cb None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:08.686790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f8da9-8c2a-4323-8cdd-79ec659721cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:08.696976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 852/1746] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:08.766684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b34f8da9-8c2a-4323-8cdd-79ec659721cb None None] MFA Rules not processed for user `9e5f6f21fcbe4aceb97069348c583c62`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:08.770501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b34f8da9-8c2a-4323-8cdd-79ec659721cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:08.790876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:08.793249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.058312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 895/1747] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:08 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:09.059593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.059941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.060225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.060413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98472043-5186-49ec-8752-d15fa0c2c95f service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:09.086772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:09.087439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.087729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.088087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] PATH_INFO: `/v3/users/97a19ec3aa234a9c8d384086fb5467cd` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.094093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a19ec3aa234a9c8d384086fb5467cd)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:09.097479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:09.104928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] MFA Rules not processed for user `b99bdb65eb5c472ab7711735f3d35a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:09.106201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.249151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a19ec3aa234a9c8d384086fb5467cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:09.254352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a19ec3aa234a9c8d384086fb5467cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:09.279718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 896/1748] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:09.283418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:09.288831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.290582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a19ec3aa234a9c8d384086fb5467cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:09.317582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a19ec3aa234a9c8d384086fb5467cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:09.330345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b63fe-169e-4204-90d4-5f97b032c1d0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:09.332597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ca4027de-c14e-4208-8e52-0bc71045bdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a19ec3aa234a9c8d384086fb5467cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:09.339623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:09.340774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.341041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.341320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.345746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 853/1749] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:08 2022] DELETE /identity/v3/users/97a19ec3aa234a9c8d384086fb5467cd => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:09.347616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.348627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b933b5-523d-4f23-8a83-eeb62fbaef09 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.348954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b933b5-523d-4f23-8a83-eeb62fbaef09 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.350587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b933b5-523d-4f23-8a83-eeb62fbaef09 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.353830 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 854/1750] 10.222.0.78 () {54 vars in 961 bytes} [Wed Aug 24 10:00:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:00:09.356013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.358381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.358803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.410632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:09.411908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.449252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:09.501009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:09.503160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3762458-a2ef-4e69-a551-4764a517cc35 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:09.518956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 897/1751] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:09.522716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.523019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.523306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.572652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:09.574398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.595503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 855/1752] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:09.596962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:09.604075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.618269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:09.708696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa133ad-ab24-4957-bad3-d67591ee358a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:09.713706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:09.714625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.715394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.715747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.719097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.780598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 898/1753] 10.222.0.78 () {58 vars in 1058 bytes} [Wed Aug 24 10:00:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:09.782789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:09.787882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.854778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:09.857450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:09.877171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 856/1754] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:09.878104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:09.895348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719123e-91d6-4f8d-8caa-7e25d6f15e4f service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:09.902965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:09.904221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.904622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.905636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:09.909063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:09.926027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:09.928333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:09.944322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 857/1755] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:09.975342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81476c96-92ed-4ba7-b12d-982940d6fbda None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:09.981670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:09.984773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:09.985208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:09.985501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:10.004479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:10.005948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb46532-368d-4cbd-a28c-c36a1c563269 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:10.017784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 899/1756] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:10.641204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:10.643239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:10.674146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2192f-7dc3-47a8-9ee4-4f59eadfe3b1 service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:10.703092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:10.708450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:10.708981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:10.709468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] PATH_INFO: `/v3/projects/a3ec5040aac04fe28c38e3add3ca4642` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:10.712723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ec5040aac04fe28c38e3add3ca4642)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:10.715773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:10.887429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ec5040aac04fe28c38e3add3ca4642', 'request_id': 'req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:10.905187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ec5040aac04fe28c38e3add3ca4642', 'request_id': 'req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:10.920895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ec5040aac04fe28c38e3add3ca4642', 'request_id': 'req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:10.942345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] The token cache is being invalidate because project a3ec5040aac04fe28c38e3add3ca4642 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:10.944329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:10.947279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d1bdd0fc-08f9-4217-b1d5-3667bc08f8d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:10.966430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 858/1757] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:10 2022] DELETE /identity/v3/projects/a3ec5040aac04fe28c38e3add3ca4642 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:10.972132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:10.973392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:10.973392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.028775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:11.030148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:11.197925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 900/1758] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:11.206443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.207178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.207635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.258039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:11.261624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:11.283930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 859/1759] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:11.289981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:11.308350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a57ddf-b620-415b-b7e4-b1c6d3330f40 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:11.315750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:11.316523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.317013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.317409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.318274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:11.318712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:11.320631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:11.360049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 901/1760] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:11.363707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:11.376394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86519697-1ad1-4dbd-9956-02346deb9f7b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:11.399577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:11.402889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.403409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.403920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.405762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:11.409881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:11.482581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:11.516947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 860/1761] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:11 2022] POST /identity/v3/projects => generated 363 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:11.521036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:11.536748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822c1a4-68c4-43f5-a504-f47b209f1306 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:11.555757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:11.722854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:11.723913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.726226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.726724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.727589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:11.729810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:11.803498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:11.813626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 902/1762] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:11 2022] POST /identity/v3/users => generated 282 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:11.820753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eff5ba-4698-47e3-b3aa-5e7566ae47cd admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:11.844494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:11.844494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.844760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.845012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.848740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:11.850350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:11.906639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:11.916843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 861/1763] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:11.935361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32969020-86a1-4a3a-bdb8-982e3d1bc5e3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:11.969562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:11.970289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:11.970581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:11.970862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:11.971530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:11.973310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:12.020128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.032311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 903/1764] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:11 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:12.038937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9cccf-1bdb-463d-ab14-a73806d721b9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:12.059536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:12.060522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.060967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.061656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] PATH_INFO: `/v3/projects/09a21841ec1344798ac58f5f1d709641/users/6ce08d954c5f4962ac8e9639f3182c38/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.086809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.091630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.113688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a21841ec1344798ac58f5f1d709641, user_id=6ce08d954c5f4962ac8e9639f3182c38, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:12.120492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dba541-9e13-4001-aa1a-9f5696861ff6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:12.121061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f833069-7061-45db-859c-366c9c80764c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:12.145296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:12.148836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.151259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.151836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] PATH_INFO: `/v3/users/abaa343237d34a3099880f315a699125` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.155418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abaa343237d34a3099880f315a699125)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:12.157718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:12.189937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 862/1765] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:12 2022] PUT /identity/v3/projects/09a21841ec1344798ac58f5f1d709641/users/6ce08d954c5f4962ac8e9639f3182c38/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:12.192642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.192940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.193212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.253211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] MFA Rules not processed for user `6ce08d954c5f4962ac8e9639f3182c38`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:12.255301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.326020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abaa343237d34a3099880f315a699125'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:12.326778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abaa343237d34a3099880f315a699125'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:12.352207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abaa343237d34a3099880f315a699125'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:12.386933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abaa343237d34a3099880f315a699125'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:12.410807 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-48dbcc92-e243-48ab-8757-cdd9bdd1fa94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abaa343237d34a3099880f315a699125'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:12.429936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 904/1766] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:12 2022] DELETE /identity/v3/users/abaa343237d34a3099880f315a699125 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:12.443328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.453728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.454008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.465273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 863/1767] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:12.470876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.474336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.513624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:12.515964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.611907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf054f9-538f-4cf7-8ada-e99961d61f5d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:12.616273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:12.648312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 905/1768] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:12.655142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.657554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.718440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:12.730143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:12.731030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.732468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.732604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.733559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:12.736365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:12.751609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e14727-663a-43f6-b2b0-a3dbccc1a712 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:12.758161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:12.758895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.759105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.759385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.773475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:12.775186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:12.775842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 864/1769] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:12 2022] POST /identity/v3/projects => generated 363 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:12.779843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.791914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 906/1770] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:12.951485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7f974-ae6c-45be-8e1c-3cd96ada88d3 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:12.981968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:12.985131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:12.987049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:12.990287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:12.991439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:12.993304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:12.995968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.004061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.006721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952b9ba3-152c-47ce-9fed-66fcc8fb63a9 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.016874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.017599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.018039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.018407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] PATH_INFO: `/v3/users/122de83caea74b80b6033714f0c88a1b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.019611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122de83caea74b80b6033714f0c88a1b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.022190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.126185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 865/1771] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:12 2022] POST /identity/v3/users => generated 288 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:13.130176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.149189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122de83caea74b80b6033714f0c88a1b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.152313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122de83caea74b80b6033714f0c88a1b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.156033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec267653-9bfa-465e-af27-718ded652410 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.165963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122de83caea74b80b6033714f0c88a1b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.169150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.170577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.171646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.172543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.174028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.178305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.209699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 866/1772] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:13.215261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.216409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122de83caea74b80b6033714f0c88a1b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.220735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.239839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-861b9c35-5a28-46bc-a60f-8a23e20cf5e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122de83caea74b80b6033714f0c88a1b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.264089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 907/1773] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:12 2022] DELETE /identity/v3/users/122de83caea74b80b6033714f0c88a1b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:13.270123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.270554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.271481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.274992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb72de8-3aff-45a5-848a-8d25ac9cce19 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.335888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:13.336089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.486777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 908/1774] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:13.487848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.489533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.522649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de23bab4-61d3-485b-a7fd-03c406bf5221 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.522808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.523917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.524448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.524933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] PATH_INFO: `/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c/users/abd13157414546f085f2558514f25ae6/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.530087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.531577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.532097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.532450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] PATH_INFO: `/v3/projects/3e9c6d1cab3447f6b1a8aad4893ef281` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.533748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e9c6d1cab3447f6b1a8aad4893ef281)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.535824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.544250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8d2fda3dd54e6f85fc85bd2401643c, user_id=abd13157414546f085f2558514f25ae6, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.548707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.561509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:13.595904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 867/1775] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:13 2022] PUT /identity/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c/users/abd13157414546f085f2558514f25ae6/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:13.595904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.599932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.647963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e9c6d1cab3447f6b1a8aad4893ef281', 'request_id': 'req-9298147f-5618-4c53-b3a6-378860067dfd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.673375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e9c6d1cab3447f6b1a8aad4893ef281', 'request_id': 'req-9298147f-5618-4c53-b3a6-378860067dfd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.678431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:13.690949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e9c6d1cab3447f6b1a8aad4893ef281', 'request_id': 'req-9298147f-5618-4c53-b3a6-378860067dfd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.695301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdbbc5-0f2b-4279-b06f-87a96eaa7933 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.708923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] The token cache is being invalidate because project 3e9c6d1cab3447f6b1a8aad4893ef281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:13.709216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.709921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:13.716333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 909/1776] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:13 2022] DELETE /identity/v3/projects/3e9c6d1cab3447f6b1a8aad4893ef281 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:13.717270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.719521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.806980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.807825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.808248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.808534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.814114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.830431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298147f-5618-4c53-b3a6-378860067dfd admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.836582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.838632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.839164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.839841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.840802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.843721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.847137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.848926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.867296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 910/1777] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:13.867296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.869536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:13.879532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 868/1778] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:13.881224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:13.896176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9ef6-3b21-49f0-a572-fa3cfe98773f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.899533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154a2a-6f06-4e76-a622-2cffd90926eb service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:13.914311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.915003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.916004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:13.916657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.916933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.917205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] PATH_INFO: `/v3/users/9e5f6f21fcbe4aceb97069348c583c62` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.917994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.918284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] PATH_INFO: `/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c/users/abd13157414546f085f2558514f25ae6/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:13.920480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5f6f21fcbe4aceb97069348c583c62)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.922897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.929568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8d2fda3dd54e6f85fc85bd2401643c, user_id=abd13157414546f085f2558514f25ae6, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:13.938626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763324a-bd8d-48f8-924f-4c8ac8deb29c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:13.966721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 869/1779] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:13 2022] PUT /identity/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c/users/abd13157414546f085f2558514f25ae6/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:13.978236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:13.978543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:13.978846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.036347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5f6f21fcbe4aceb97069348c583c62'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.036347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5f6f21fcbe4aceb97069348c583c62'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.053791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5f6f21fcbe4aceb97069348c583c62'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.057565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] MFA Rules not processed for user `abd13157414546f085f2558514f25ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:14.058799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.080924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5f6f21fcbe4aceb97069348c583c62'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.095051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5f6f21fcbe4aceb97069348c583c62'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.120206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 911/1780] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:13 2022] DELETE /identity/v3/users/9e5f6f21fcbe4aceb97069348c583c62 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:14.133262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.134731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.141908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:14.169132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 870/1781] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:14.182967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.185974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.227243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e454887-4b5e-4876-a255-d127df0d4061 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.236155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.237057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.237626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.238147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] PATH_INFO: `/v3/users/d05eb809bed54d34be15c63eaf56dd99` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.242354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d05eb809bed54d34be15c63eaf56dd99)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.246832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.275201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8736e-1a56-4546-a3d2-875d595b26d5 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.307281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd05eb809bed54d34be15c63eaf56dd99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.308069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd05eb809bed54d34be15c63eaf56dd99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.319011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd05eb809bed54d34be15c63eaf56dd99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.325324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:14.348776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd05eb809bed54d34be15c63eaf56dd99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.365447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.366404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.366905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.367523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.370350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd05eb809bed54d34be15c63eaf56dd99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.370784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.387618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 912/1782] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:14 2022] DELETE /identity/v3/users/d05eb809bed54d34be15c63eaf56dd99 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:14.389140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.390988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.479379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.485952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25146b35-340b-480f-a6f4-aa0eff8f3acb service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.492311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae770e2-9e3c-4573-84ba-c3f6e45648a6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.498546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.499333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.500203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.500510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] PATH_INFO: `/v3/projects/1d18d5eb6d1b4b598f5ed177cdd2f25b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.501885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d18d5eb6d1b4b598f5ed177cdd2f25b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.504465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.519373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 871/1783] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:00:14.521633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.522036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.522520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.549005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:14.550744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.553785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:14.562786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 872/1784] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:14.576432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.578601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.623762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d18d5eb6d1b4b598f5ed177cdd2f25b', 'request_id': 'req-24ee7349-bf90-4259-946c-c01f253c6d7c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.638325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d18d5eb6d1b4b598f5ed177cdd2f25b', 'request_id': 'req-24ee7349-bf90-4259-946c-c01f253c6d7c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.652798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d18d5eb6d1b4b598f5ed177cdd2f25b', 'request_id': 'req-24ee7349-bf90-4259-946c-c01f253c6d7c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.676281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f853a28-248c-4f1c-bd61-da67b96e28a7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.676580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] The token cache is being invalidate because project 1d18d5eb6d1b4b598f5ed177cdd2f25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:14.677083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.677867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.680087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 913/1785] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:14 2022] DELETE /identity/v3/projects/1d18d5eb6d1b4b598f5ed177cdd2f25b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:14.680901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.682533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.756021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.756021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.756021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.756021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.756346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.771868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee7349-bf90-4259-946c-c01f253c6d7c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.777662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.778199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.778545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.778885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] PATH_INFO: `/v3/projects/acf6cb08467f49f399a45342c4fe1260` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.780177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acf6cb08467f49f399a45342c4fe1260)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.782647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.802416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:14.823069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.824561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.830944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acf6cb08467f49f399a45342c4fe1260', 'request_id': 'req-b784cd24-b887-475e-a51c-1479e3923740'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.835465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 873/1786] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:14.836378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.838062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.841444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acf6cb08467f49f399a45342c4fe1260', 'request_id': 'req-b784cd24-b887-475e-a51c-1479e3923740'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.848570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acf6cb08467f49f399a45342c4fe1260', 'request_id': 'req-b784cd24-b887-475e-a51c-1479e3923740'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.860865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] The token cache is being invalidate because project acf6cb08467f49f399a45342c4fe1260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:14.861082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.861883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:14.868215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 914/1787] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:14 2022] DELETE /identity/v3/projects/acf6cb08467f49f399a45342c4fe1260 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:14.868215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:14.868215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:14.929973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23dcc9-5687-40ab-8832-ed6adc3ef5f9 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.935236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:14.968794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b784cd24-b887-475e-a51c-1479e3923740 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:14.973890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.974531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.974825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.976170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] PATH_INFO: `/v3/users/98b29975d9d2438aaf4a0eb85b266a2d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:14.976340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] RBAC: Authorizing `identity:delete_user(user_id=98b29975d9d2438aaf4a0eb85b266a2d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:14.978355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:14.997300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:14.998008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:14.998545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:14.998545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.001788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.049629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98b29975d9d2438aaf4a0eb85b266a2d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.053003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98b29975d9d2438aaf4a0eb85b266a2d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.070844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98b29975d9d2438aaf4a0eb85b266a2d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.090443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98b29975d9d2438aaf4a0eb85b266a2d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.098973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.100526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d1ed2-fb01-47ee-b94a-c55e4ba9a7bb service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.103190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-75398ddf-ba80-4e5a-82a7-ac256bfe1eab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98b29975d9d2438aaf4a0eb85b266a2d'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.111072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 915/1788] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:14 2022] DELETE /identity/v3/users/98b29975d9d2438aaf4a0eb85b266a2d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:15.112283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 874/1789] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:15.112912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.113209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.113489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.136116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:15.137367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.211413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 916/1790] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:15.221196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.221688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.221913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.242842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.246084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.246206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:15.247712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.269225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61f78f-8c72-404d-bb74-06f7dbf8b9d3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.277380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.277863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 875/1791] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:15.283962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.284479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.285294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.285750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] PATH_INFO: `/v3/projects/5cdcf98f95004bc081ad3ebd73124084` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.287736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cdcf98f95004bc081ad3ebd73124084)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.291083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.311025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd7897-4057-4e90-a854-5b47d6b3888d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.326850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.327641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.329662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.330123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.332175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:15.332867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.341334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.367930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 876/1792] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:15.370729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.379408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cdcf98f95004bc081ad3ebd73124084', 'request_id': 'req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.387303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cc2a65-c989-4131-9ebb-55ab084af403 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.393666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cdcf98f95004bc081ad3ebd73124084', 'request_id': 'req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.397194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:15.404206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cdcf98f95004bc081ad3ebd73124084', 'request_id': 'req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.426628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] The token cache is being invalidate because project 5cdcf98f95004bc081ad3ebd73124084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:15.426824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:15.427084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.428100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.430005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 917/1793] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:15 2022] DELETE /identity/v3/projects/5cdcf98f95004bc081ad3ebd73124084 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:15.434033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.435219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.478083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.478862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.479216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.479563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.480329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.482622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.537456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 877/1794] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:15.538171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8c2b1-df8d-49e1-b881-a8e8225cdb5b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.538551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.540537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.628682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.629248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.629527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.629800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] PATH_INFO: `/v3/users/7cb40b416d7546f187fc5ecea6a3eff2` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.631232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb40b416d7546f187fc5ecea6a3eff2)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.633003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac8a7-0fa1-4ff9-98b2-a860263a684d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.633379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.639426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.640828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.641275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.641708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.642503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.644694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.677913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 878/1795] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/users => generated 290 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:15.679162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.681378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.698664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc6b6d-7b86-46ac-8425-4940103eda7d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.703824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.704528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.704813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.705088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] PATH_INFO: `/v3/projects/ef79745c99e845fc993df1fbcdeaa71a` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.706345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef79745c99e845fc993df1fbcdeaa71a)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.708879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.719395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb40b416d7546f187fc5ecea6a3eff2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.720805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb40b416d7546f187fc5ecea6a3eff2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.731410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb40b416d7546f187fc5ecea6a3eff2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.751576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb40b416d7546f187fc5ecea6a3eff2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.764203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb40b416d7546f187fc5ecea6a3eff2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.772994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 918/1796] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:15 2022] DELETE /identity/v3/users/7cb40b416d7546f187fc5ecea6a3eff2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:15.774114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.775594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.780657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef79745c99e845fc993df1fbcdeaa71a', 'request_id': 'req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.802965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef79745c99e845fc993df1fbcdeaa71a', 'request_id': 'req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.816022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef79745c99e845fc993df1fbcdeaa71a', 'request_id': 'req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.816831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:15.834735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] The token cache is being invalidate because project ef79745c99e845fc993df1fbcdeaa71a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:15.835816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.836772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e1dd1ae3-b106-43e7-8d95-8156d7fde97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:15.838907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 879/1797] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:15 2022] DELETE /identity/v3/projects/ef79745c99e845fc993df1fbcdeaa71a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:15.846351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cceae-b4a1-4b38-89fc-71b67b84d8e1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.846816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cceae-b4a1-4b38-89fc-71b67b84d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.846992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cceae-b4a1-4b38-89fc-71b67b84d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.872170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ec2cceae-b4a1-4b38-89fc-71b67b84d8e1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:15.873448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ec2cceae-b4a1-4b38-89fc-71b67b84d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.905588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d960d6-0d98-4498-8e33-2486de8408ff admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.910950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:15.912302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.913257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.913624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.914114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.915465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.917618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.945617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 919/1798] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:15.950594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:15.952832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:15.960775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 880/1799] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:15.969618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10da3c-8b6b-476e-9f0c-bbbc3cba7849 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:15.970812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.971147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.971433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.975576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:15.976458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:15.976914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:15.977411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:15.978075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:15.979964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:15.998059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:15.999404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 920/1800] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:16.004176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.004374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.017744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 881/1801] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:16.022442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.024552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec7b90-e545-4d1e-b28e-8ac5270506f7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.029726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.030394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.030679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.031541 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] PATH_INFO: `/v3/projects/e28a3b8520844694adf0244e458a3b8d/users/d583f50b5e5641db87e68d9f41388b26/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.036086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb7c69-bd52-4ec0-8bb3-698669723f2b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.040875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.041644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.041933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.042204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.043024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:16.043194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.044803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.048754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28a3b8520844694adf0244e458a3b8d, user_id=d583f50b5e5641db87e68d9f41388b26, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.051619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbffca0-b1f2-4a85-9162-4f332ceac127 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.058523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 882/1802] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:00:16.063625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.077769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a6b71e-b2dd-4507-aaca-72f91fc3f019 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.098756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 921/1803] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:16 2022] PUT /identity/v3/projects/e28a3b8520844694adf0244e458a3b8d/users/d583f50b5e5641db87e68d9f41388b26/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:16.113374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.113662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.113939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.164039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.164988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.165341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.165690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.166435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.168556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.170601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] MFA Rules not processed for user `d583f50b5e5641db87e68d9f41388b26`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:16.172995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.209528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 883/1804] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/projects => generated 387 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:16.211941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.267569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 922/1805] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:16.272555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.293259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93280146-6a40-492d-af7b-eb6a8d27c11e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.293455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32526a-ed52-4459-8499-e3bec970c43d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.299229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.300729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.300993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.301268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.301909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.304060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.304926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.305975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.306552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.307284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.308704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.320774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.341890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 884/1806] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:16.343391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.367716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2021e64-86eb-4385-a2ba-7c629cc193c8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.381105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 923/1807] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:16.397394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.411547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.413460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.414336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.414532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.414838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.415490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.417349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.455955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 885/1808] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:16.455955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.472469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b902453-603c-49ee-b25b-4af8c3a0666c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.490929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:16.544613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:16.571360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccec10-63c3-4d98-aff6-35a61ab40c6c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.572956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.573900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.574198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.574656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.575358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.577641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.578628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.580540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.580916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.581274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] PATH_INFO: `/v3/projects/d7167a36020948898c747c672aee1aad` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.582725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7167a36020948898c747c672aee1aad)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.585457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.613459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 886/1809] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/users => generated 290 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:16.614409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.627852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c66664-70a6-42a5-ba32-f5741639d962 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.634741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.635730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.636683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.636810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.637775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.639518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.656872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 887/1810] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:16.656872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.658438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7167a36020948898c747c672aee1aad', 'request_id': 'req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:16.669308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7167a36020948898c747c672aee1aad', 'request_id': 'req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:16.669475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864197e4-4c9b-4a56-b010-6ce22813917d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.688942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7167a36020948898c747c672aee1aad', 'request_id': 'req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:16.710677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] The token cache is being invalidate because project d7167a36020948898c747c672aee1aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:16.710994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:16.712233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:16.720981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 924/1811] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:16 2022] DELETE /identity/v3/projects/d7167a36020948898c747c672aee1aad => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:16.721249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.728103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.758593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.760325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.760489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.760766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.762241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.767301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57609e1-e4f9-4058-8e7c-fd41fab62870 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.781964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 888/1812] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:16.783491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.783776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.784297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.811702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:16.814699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.847235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b25276-83f0-45e5-a0c8-4b8d2afb4c7f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.855834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 889/1813] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:16.856696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:16.865044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.865938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.866271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.866673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] PATH_INFO: `/v3/projects/70cf9882f7214a66bb65980bbb1fd4c8/users/12e6d05796da44b5a6385e19646c0634/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.867829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:16.904582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70cf9882f7214a66bb65980bbb1fd4c8, user_id=12e6d05796da44b5a6385e19646c0634, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.907391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338e9e0-668c-46e2-98ba-7d581bd33d22 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:16.907936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483639f1-8759-4902-a4ec-5808c054146c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.916890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:16.917596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.917878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.918218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.918878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:16.920777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2336d-6602-41dc-b067-2076a88b4341 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:16.945412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 925/1815] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:16 2022] PUT /identity/v3/projects/70cf9882f7214a66bb65980bbb1fd4c8/users/12e6d05796da44b5a6385e19646c0634/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:16.945655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 890/1815] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:16.945787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.945787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.945787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.946181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9b5f-2100-4366-bf09-1eb32567d4b4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:16.946181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9b5f-2100-4366-bf09-1eb32567d4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:16.946181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d9b5f-2100-4366-bf09-1eb32567d4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:16.975784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:16.977424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:17.021514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e49d9b5f-2100-4366-bf09-1eb32567d4b4 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:17.026239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e49d9b5f-2100-4366-bf09-1eb32567d4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:17.105347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:17.114848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:17.138684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 926/1816] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:17.142844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.153246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 891/1817] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:00:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:17.163691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1f1f6-ac59-4465-930d-c3ec399b24af None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.180117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.180695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.180695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.181193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.181950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.182240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.182601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] PATH_INFO: `/v3/projects/8fb07a9c0c664f199fbe10323d3e33dd/users/a7f5a4a482204fdba3f5857fde7719f9/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.220418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb07a9c0c664f199fbe10323d3e33dd, user_id=a7f5a4a482204fdba3f5857fde7719f9, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.224106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.253241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 927/1818] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:17 2022] PUT /identity/v3/projects/8fb07a9c0c664f199fbe10323d3e33dd/users/a7f5a4a482204fdba3f5857fde7719f9/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:17.255730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.262624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] MFA Rules not processed for user `12e6d05796da44b5a6385e19646c0634`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:17.266193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:17.376724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019b1df-22a0-4e7f-a583-42c491ddc72b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.382200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.382948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.383465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.383635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.384475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:17.384846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.387182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b73e0b-5eab-4383-ad7f-d0a425e98797 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.403747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 928/1819] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:17.405178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.405543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.405941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.429171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 892/1820] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:17.430787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.431878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:17.488068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] MFA Rules not processed for user `a7f5a4a482204fdba3f5857fde7719f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:17.492279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:17.627666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 929/1821] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:17.628896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.655037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31399d5e-5dca-4c58-b7b7-76225295b515 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.661630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.662270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.662669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.663077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.663799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.666012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.685509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d561304e-9287-42c2-9bdb-eace7e72dd2f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.693458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.694531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.696080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.696669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.710903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 930/1822] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/projects => generated 409 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:17.711829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.768069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:17.769239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:17.770469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.772566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.800107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 893/1823] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:00:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:00:17.801443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.821806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54871959-42d7-4935-92d5-719e5378c5bc admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.825407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7cb1d-0c23-4760-985d-344b3603a113 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:17.828108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.828545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.828829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.829108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.830461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.832475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0351ee5-0172-4f57-9101-5ea5e621e61b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.847722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:17.852467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.852757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.854686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.856021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:17.867536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:17.881637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 931/1824] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:17.884545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:17.886215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:17.888774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:17.940062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 894/1825] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:17.940062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:17.945611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:17.949386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:18.108833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 932/1826] 10.222.0.84 () {58 vars in 1015 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:18.110555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.120309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3398d04d-5d5d-4075-bf73-7eb8136d8b03 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.134003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.138344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.139327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.141841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.143180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.153496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ce7b9-44a1-4b21-b7ef-c6e65b9e3021 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.164946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.174992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.177782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.178220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.178591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.182856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.184954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.256817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 933/1827] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:18 2022] POST /identity/v3/users => generated 300 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:18.258443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.281880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082f97d-a208-4fdc-8f62-b4f9d8b37a60 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.297736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 895/1828] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:17 2022] POST /identity/v3/users => generated 305 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:18.299071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.301721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.302564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.303663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:18.306141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.306464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.308410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.316200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.339485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9dc88-8829-4d2f-8d17-2cb7e7927176 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.364408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.364408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.364408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.364408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.383558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.386625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.395844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 934/1829] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:18 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:18.400587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.420951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 896/1830] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:18.428764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.438314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc7d07-4b6c-403a-ac05-4d382bf045c0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.465660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.466366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.466569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.467151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.467846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.470065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.502724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdad1-5253-4783-befa-4805190242fa service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.521378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 935/1831] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:18.522810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.539177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.541857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.544499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.544863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.552325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.560916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.561533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0d2c-1141-4c0e-81e1-253efe63766c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.587658 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.588646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.589357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.589827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.590646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.592537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.597318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 897/1832] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:18.598173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.615426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 936/1833] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:18.617378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.627335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b9967-02e2-4742-8c00-be6f865d9b31 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.646257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d852913-4d1c-4759-ab36-01066df3bfc5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.650442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.651327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.651763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.652448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] PATH_INFO: `/v3/projects/c706f8dd538843aa88bd1f94921cb17c/users/6ed7038041b0474ba149a1f6ff288ffa/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.676510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.677177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.677496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.677781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.678488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.680148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.733193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c706f8dd538843aa88bd1f94921cb17c, user_id=6ed7038041b0474ba149a1f6ff288ffa, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.737597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 937/1834] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:18.740850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.748852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.778431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3173b-d5f8-4bad-8d39-665092c0dea7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:18.805851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:18.806550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:18.806836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:18.807113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] PATH_INFO: `/v3/projects/06ddd06f4fb847c0b6dedbef26d27321/users/acb33ce183894caca6bfe6327e0309f3/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:18.816801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 898/1835] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:18 2022] PUT /identity/v3/projects/c706f8dd538843aa88bd1f94921cb17c/users/6ed7038041b0474ba149a1f6ff288ffa/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:18.823093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:18.861659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ddd06f4fb847c0b6dedbef26d27321, user_id=acb33ce183894caca6bfe6327e0309f3, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:18.864565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:18.916215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 938/1836] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:18 2022] PUT /identity/v3/projects/06ddd06f4fb847c0b6dedbef26d27321/users/acb33ce183894caca6bfe6327e0309f3/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:18.917666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:19.054290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:19.065242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef0b81e-41d4-4b01-a970-b39f3f2d5a29 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:19.096669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e3ac-23a5-40e4-9d6e-2e33aa1f2f29 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:19.102567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:19.104359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:19.105176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.106945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.107238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.107518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.108496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.108618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:19.110346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] PATH_INFO: `/v3/projects/1b6ac29e18f94f81883f8f70d674d737/users/b7ca3a7cb8604c4e9b22b01605e75b6b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.115435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:19.155532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 939/1837] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:19.155532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:19.193670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dff5f3-5754-4bf7-98f3-7a9e9c4a1bf0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:19.196837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b6ac29e18f94f81883f8f70d674d737, user_id=b7ca3a7cb8604c4e9b22b01605e75b6b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:19.204767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:19.206874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.208498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.208802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.209633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:19.210157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:19.217950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:19.221774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:19.248797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 940/1838] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:19 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:19.248797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:19.254120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 899/1839] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:18 2022] PUT /identity/v3/projects/1b6ac29e18f94f81883f8f70d674d737/users/b7ca3a7cb8604c4e9b22b01605e75b6b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:19.258559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:19.271928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e66bf-4d65-4c32-8b1c-18cc89d20110 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:19.464146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:19.465954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.466251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.466583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] PATH_INFO: `/v3/projects/c706f8dd538843aa88bd1f94921cb17c/users/6ed7038041b0474ba149a1f6ff288ffa/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.475069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c706f8dd538843aa88bd1f94921cb17c, user_id=6ed7038041b0474ba149a1f6ff288ffa, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:19.482969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb6b24-1a5e-4eb7-859d-bdbea90feea4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:19.511209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 941/1840] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:19 2022] PUT /identity/v3/projects/c706f8dd538843aa88bd1f94921cb17c/users/6ed7038041b0474ba149a1f6ff288ffa/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:19.515257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585c43f-7479-49e5-9ad0-9d06490302e5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.516127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585c43f-7479-49e5-9ad0-9d06490302e5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.516411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585c43f-7479-49e5-9ad0-9d06490302e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.614695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3585c43f-7479-49e5-9ad0-9d06490302e5 None None] MFA Rules not processed for user `b7ca3a7cb8604c4e9b22b01605e75b6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:19.615983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3585c43f-7479-49e5-9ad0-9d06490302e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:19.626631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76cb16-d50c-4cf4-bb0e-98dad0a71d69 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:19.870937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 942/1841] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:19.878564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475731d-fc9f-4c49-99da-8e813175b6a1 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.881822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475731d-fc9f-4c49-99da-8e813175b6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.889304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475731d-fc9f-4c49-99da-8e813175b6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.900012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:19.901551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:19.903013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:19.903904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] PATH_INFO: `/v3/projects/06ddd06f4fb847c0b6dedbef26d27321/users/acb33ce183894caca6bfe6327e0309f3/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:19.911142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ddd06f4fb847c0b6dedbef26d27321, user_id=acb33ce183894caca6bfe6327e0309f3, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:19.932258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:19.995235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 900/1842] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:19 2022] PUT /identity/v3/projects/06ddd06f4fb847c0b6dedbef26d27321/users/acb33ce183894caca6bfe6327e0309f3/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:19.997367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.024665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8475731d-fc9f-4c49-99da-8e813175b6a1 None None] MFA Rules not processed for user `6ed7038041b0474ba149a1f6ff288ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:20.029126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8475731d-fc9f-4c49-99da-8e813175b6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:20.103410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:20.222542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 943/1843] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:20.233849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.237182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.237875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:20.243295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c68c30-dcee-4012-b1c8-62a9412acc4d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:20.262360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:20.263644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.265234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.266708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:20.267719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:20.275657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:20.299120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] MFA Rules not processed for user `acb33ce183894caca6bfe6327e0309f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:20.300557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:20.366090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 901/1844] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:19 2022] POST /identity/v3/projects => generated 407 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:20.367046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.434186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:20.546637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c4e60-fc5f-48b2-9cc3-566f408f1a4e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:20.564598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:20.565243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.565533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.565796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:20.566432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:20.568439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:20.614588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 944/1845] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:20.617589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.622121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:20.636351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 902/1846] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:20 2022] POST /identity/v3/users => generated 310 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:20.637692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.660966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621203e6-5e83-4363-a998-c68a3ae5ed86 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:20.676772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:20.677599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.678067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.678512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:20.679719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:20.681556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:20.723912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 903/1847] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:20.725174 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.751768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c8bdc-1ecc-4ee1-9482-cffceda22ef1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:20.766333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:20.767354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.767950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.768540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] PATH_INFO: `/v3/projects/1088d03104924c78834aff933f86a686/users/fd0c9d872c4e47f9b84eef89460c2472/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:20.805108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1088d03104924c78834aff933f86a686, user_id=fd0c9d872c4e47f9b84eef89460c2472, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:20.808287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:20.841123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 904/1848] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:20 2022] PUT /identity/v3/projects/1088d03104924c78834aff933f86a686/users/fd0c9d872c4e47f9b84eef89460c2472/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:20.844195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:20.870978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388b9d9-1760-4635-9e68-f8c38cf9ace6 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:20.968831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:20.970067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:20.970355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:20.970644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.014872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd7957-c302-45e0-bd43-6a1f26e51890 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.025310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.025944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.027216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.027500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.028182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.032732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebced22-42c6-4a65-9ed6-9fcf3bd4bf4e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.042503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:21.069278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 905/1849] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:20 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:21.088504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bc9-ba64-4b4f-bbb8-a3f51dc7e411 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.089289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bc9-ba64-4b4f-bbb8-a3f51dc7e411 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.089581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb9bc9-ba64-4b4f-bbb8-a3f51dc7e411 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.092523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 906/1850] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:00:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:00:21.093629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb9bc9-ba64-4b4f-bbb8-a3f51dc7e411 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:21.139958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb9bc9-ba64-4b4f-bbb8-a3f51dc7e411 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.165811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.166655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.167343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644b27b6-fe46-46d6-bff3-13c465741f2c service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.170089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.170388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.170671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] PATH_INFO: `/v3/projects/1088d03104924c78834aff933f86a686/users/fd0c9d872c4e47f9b84eef89460c2472/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.176078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1088d03104924c78834aff933f86a686, user_id=fd0c9d872c4e47f9b84eef89460c2472, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.183721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.195145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 945/1851] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:21.196491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf0508-424d-4b74-abef-f3a17761cb07 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.196774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf0508-424d-4b74-abef-f3a17761cb07 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.197060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cf0508-424d-4b74-abef-f3a17761cb07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.211726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 907/1852] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:21 2022] PUT /identity/v3/projects/1088d03104924c78834aff933f86a686/users/fd0c9d872c4e47f9b84eef89460c2472/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:21.213234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:21.224430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:21.253164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-85cf0508-424d-4b74-abef-f3a17761cb07 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:21.257769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-85cf0508-424d-4b74-abef-f3a17761cb07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:21.408442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087d406-ea4a-451c-bc33-f9565957b487 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.417061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.418630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.420171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.420543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.448099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:21.467861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 946/1853] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:21.470066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.470419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.470704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.514418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] MFA Rules not processed for user `fd0c9d872c4e47f9b84eef89460c2472`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:21.517308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:21.598208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.599336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.618280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 908/1854] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:21.634741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:21.638905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:21.687388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7919c-03c8-48c2-b692-caa061024e53 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.696481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.697120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.697409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.697802 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.711163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.713400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.730446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 909/1855] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:21.744118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 947/1856] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:21 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:21.814928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:21.973001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:21.975773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e82cddd-fd39-4c52-913d-549ad11d97cf service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.983989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.985592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.985991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.986518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] PATH_INFO: `/v3/users/12e6d05796da44b5a6385e19646c0634` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:21.987880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811737d-e665-4499-8cbd-112a9490d7ee None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:21.988133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e6d05796da44b5a6385e19646c0634)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:21.993480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:21.996460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:21.997756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:21.998193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:21.998607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.001601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.105221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:22.150699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e6d05796da44b5a6385e19646c0634'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:22.151745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e6d05796da44b5a6385e19646c0634'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:22.158657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:22.161915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabb47b-5b67-4907-b3e6-220420f98d55 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:22.170776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e6d05796da44b5a6385e19646c0634'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:22.183714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e6d05796da44b5a6385e19646c0634'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:22.196465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e6d05796da44b5a6385e19646c0634'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:22.200215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 948/1857] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:00:22.211933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 910/1858] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:21 2022] DELETE /identity/v3/users/12e6d05796da44b5a6385e19646c0634 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:22.216112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.217148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.217148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.240414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:22.242095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.248041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:22.249682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.282401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:22.365045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a12614-55a8-4f17-b083-7af8d0c650e7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:22.366081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 949/1859] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:22.377702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:22.379546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.396235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:22.396890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.397261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.397494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.400070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.507184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:22.509418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e076d-4e39-4074-8595-67e6f77d663e service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:22.530073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62fef2-86f5-4bac-ba63-5c439cdc27d3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:22.534410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 911/1860] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:22.536824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:22.537817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.538260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.538731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.556256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:22.557940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9707e-6124-4d4e-a994-2f4f5bb4905b service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:22.579560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 950/1861] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:22.636723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5948-d62f-4387-bf9a-87e266b7efc1 None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.638083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5948-d62f-4387-bf9a-87e266b7efc1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.638083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65c5948-d62f-4387-bf9a-87e266b7efc1 None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.641672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 912/1862] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:00:22.650723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.651689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.652247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.685597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:22.688398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.818145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 951/1863] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:00:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:22.820056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c5948-d62f-4387-bf9a-87e266b7efc1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:22.831148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:22.833790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.835479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65c5948-d62f-4387-bf9a-87e266b7efc1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:22.842375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:22.843195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.844178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.844819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.847878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.848904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba3df0-7c3f-443b-aa2f-3cf8f528f064 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:22.855496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:22.856390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:22.856680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:22.857195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:22.872415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:22.874115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:22.882871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:22.892421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 952/1864] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:22.954378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:22.955180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:22.956873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:22.958317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:22.976544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 913/1865] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:22 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:00:22.987264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a7cc6-9b41-46ff-87e3-9ce5c6a71e1e service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:23.005458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:23.006245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:23.006537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:23.006853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] PATH_INFO: `/v3/projects/70cf9882f7214a66bb65980bbb1fd4c8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:23.009355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70cf9882f7214a66bb65980bbb1fd4c8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:23.012343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:23.068108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70cf9882f7214a66bb65980bbb1fd4c8', 'request_id': 'req-daba39bc-fb2e-4298-a2c3-084509aa9949'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.076766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70cf9882f7214a66bb65980bbb1fd4c8', 'request_id': 'req-daba39bc-fb2e-4298-a2c3-084509aa9949'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.085257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70cf9882f7214a66bb65980bbb1fd4c8', 'request_id': 'req-daba39bc-fb2e-4298-a2c3-084509aa9949'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.097984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] The token cache is being invalidate because project 70cf9882f7214a66bb65980bbb1fd4c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:23.098500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.100232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.102143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 953/1866] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:22 2022] DELETE /identity/v3/projects/70cf9882f7214a66bb65980bbb1fd4c8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:23.105903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:23.118779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cafb8e-bb74-46f1-bdae-6f3365484114 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:23.267423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:23.268225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:23.268580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:23.268886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] PATH_INFO: `/v3/users/6ed7038041b0474ba149a1f6ff288ffa` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:23.278105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ed7038041b0474ba149a1f6ff288ffa)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:23.280758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:23.289269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:23.396876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ed7038041b0474ba149a1f6ff288ffa'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.397967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ed7038041b0474ba149a1f6ff288ffa'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.419774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ed7038041b0474ba149a1f6ff288ffa'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.432754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ed7038041b0474ba149a1f6ff288ffa'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.441131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a505d456-9ebf-43c9-9417-d9fb142ea953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ed7038041b0474ba149a1f6ff288ffa'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.448100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 914/1867] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:23 2022] DELETE /identity/v3/users/6ed7038041b0474ba149a1f6ff288ffa => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:23.744585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:23.746991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:23.784258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daba39bc-fb2e-4298-a2c3-084509aa9949 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:23.902016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:23.903004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:23.903287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:23.903583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] PATH_INFO: `/v3/projects/c706f8dd538843aa88bd1f94921cb17c` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:23.905084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c706f8dd538843aa88bd1f94921cb17c)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:23.907711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:23.969195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c706f8dd538843aa88bd1f94921cb17c', 'request_id': 'req-e1c0cd45-84be-468f-a66a-eb131ddf0472'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.982182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c706f8dd538843aa88bd1f94921cb17c', 'request_id': 'req-e1c0cd45-84be-468f-a66a-eb131ddf0472'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:23.995906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c706f8dd538843aa88bd1f94921cb17c', 'request_id': 'req-e1c0cd45-84be-468f-a66a-eb131ddf0472'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:24.010371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] The token cache is being invalidate because project c706f8dd538843aa88bd1f94921cb17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:24.010702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:24.011347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e1c0cd45-84be-468f-a66a-eb131ddf0472 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:24.013094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 954/1868] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:23 2022] DELETE /identity/v3/projects/c706f8dd538843aa88bd1f94921cb17c => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:24.018801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.019438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.019438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.040191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:24.041466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:24.111754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 915/1869] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:24.120547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.120836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.121114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.143150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:24.144411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:24.155957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 955/1870] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:24.157550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.168322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137f705-cd34-4f22-8c94-d48ef234c8de None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.174257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.174911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.175193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.175467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.176150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:24.176456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.178080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.193854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 916/1871] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:24.198843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.206292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:24.212908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d4996-5920-4fec-915a-d22984dee42b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.226724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.227191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.227468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.227737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.228424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.230331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.267859 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 956/1872] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:24.272987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.284241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c0930-d9e0-46ee-bac9-1d21429fb485 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.361277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.361896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.362175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.362447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.363094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.365463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.387189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:24.394848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 917/1873] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:24.399301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.412909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75440810-2296-4bf5-8ec6-c9cba2fefca3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.417785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.418535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.418815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.419085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.419730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.421438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.434602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 957/1874] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:24.439380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.468024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681652-0af2-4f9b-808b-e86799b8492a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.481271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.486372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.486704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.486983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.488517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.490450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f939ac-cd76-49b3-bc68-1a22e9382daa admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.506512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 918/1875] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:24.510832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.529922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f0b62e-be95-4fb6-b1e9-4e538205c328 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.545682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.546372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.546653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.547066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] PATH_INFO: `/v3/projects/cda1ec7bd11343aa94ee32bfbce01fea/users/1c02d99a16d642d1a3bf015459e9d483/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.564590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda1ec7bd11343aa94ee32bfbce01fea, user_id=1c02d99a16d642d1a3bf015459e9d483, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.570512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.600017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 958/1876] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:24 2022] PUT /identity/v3/projects/cda1ec7bd11343aa94ee32bfbce01fea/users/1c02d99a16d642d1a3bf015459e9d483/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:24.618123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.618676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.618952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.660200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] MFA Rules not processed for user `1c02d99a16d642d1a3bf015459e9d483`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:24.661731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:24.741831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 919/1877] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:24.746470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.815154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62df1a-33df-4cfd-8207-f05c1477ba91 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.819905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.820818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.821631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.821631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.822209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.824158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.857591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 959/1878] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:24.858651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.873426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22333ab8-6373-44f8-91eb-36253373c5cd None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:24.945874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:24.946941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:24.947232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:24.947512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:24.948333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:24.950329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:24.983978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 920/1879] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:24 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:24.986067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:24.997660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7555bd8-f3c5-4635-8ccf-f3bd1cd7f603 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.004410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.005160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.005647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.005993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.006706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.008458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.024229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 960/1880] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:25.025338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.037975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cea63b9-9a42-4ba4-8766-19fd867f5248 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.046350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.046350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.046350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.046350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.046748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.049634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2e312-02b8-4967-a629-889604e81af3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.061593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 921/1881] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:25.066058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.076197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a7c40-d74d-4266-9cf2-3a26e5fd1fdf admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.081399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.082050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.082355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.082627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] PATH_INFO: `/v3/projects/428dd007ad6d456ab255695f3cbe1f0e/users/f12353c394264ba2a1ff67a4eedaae8c/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.097584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=428dd007ad6d456ab255695f3cbe1f0e, user_id=f12353c394264ba2a1ff67a4eedaae8c, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.100298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.113442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 961/1882] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:25 2022] PUT /identity/v3/projects/428dd007ad6d456ab255695f3cbe1f0e/users/f12353c394264ba2a1ff67a4eedaae8c/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:25.124603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.124889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.125342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.154306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] MFA Rules not processed for user `f12353c394264ba2a1ff67a4eedaae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:25.155526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:25.202161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:25.226425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 922/1883] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:25.231351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.293459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:25.298280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4adee8-f4e2-4d51-9e88-50149a9d54d8 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.306106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.306877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.307479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.307930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.308977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.311449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.352622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 962/1884] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:25 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:25.356568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.368609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc8ffbc-6c41-4fd0-a297-1ae4430c1536 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.463892 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.464553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.464868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.465100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.465809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.468039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.497788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 923/1885] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:25 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:25.501619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.518240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789844a-f974-4b1b-9bdf-d469125dc0c6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.524055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.524729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.525015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.525532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.526281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.528309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.541485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 963/1886] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:25.545951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.559905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705bcc5-a421-40cc-a737-8f0ecede3d28 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.566058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.566735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.567013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.567328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] PATH_INFO: `/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db/users/6cc5ffc217554185983a4918b166dce0/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.587193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a513a45f9453da7a7d9d8de8ad2db, user_id=6cc5ffc217554185983a4918b166dce0, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.589922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.605689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 924/1887] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:25 2022] PUT /identity/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db/users/6cc5ffc217554185983a4918b166dce0/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:25.609885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.621167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c26c1-9201-4e1a-93f6-bc2d2f3818d1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.694707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.695620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.695948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.696213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.696869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.698479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26af9d-f652-47ea-9e56-a95004e7df94 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.712505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 964/1888] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:25.717469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:25.743394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155ce1bf-7c47-48f8-b240-a4f6c5b47200 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:25.765914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:25.766836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.767064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.767436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] PATH_INFO: `/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db/users/6cc5ffc217554185983a4918b166dce0/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.769592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a22a513a45f9453da7a7d9d8de8ad2db, user_id=6cc5ffc217554185983a4918b166dce0, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:25.780323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:25.806775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 925/1889] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:25 2022] PUT /identity/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db/users/6cc5ffc217554185983a4918b166dce0/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:25.821398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:25.821957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:25.822039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:25.863744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] MFA Rules not processed for user `6cc5ffc217554185983a4918b166dce0`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:25.865670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:25.995375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 965/1890] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:26.008210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:26.011123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:26.112745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d05df8-8380-4724-a0e1-7f9b73794030 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:26.117106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:26.117819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:26.118176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:26.118524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:26.129906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:26.132389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:26.143848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 926/1891] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:26.746983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:26.759837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8f298-36df-4320-87a0-b7d97b82b662 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:26.765149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:26.767245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:26.767545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:26.767846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:26.825860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:26.837468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:26.839497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97788119-bea4-45d1-8929-25b7a51460e0 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:26.850355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 966/1892] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:36.138888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:36.845838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:38.541653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:38.543210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:38.610435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795e7be-32d2-4a31-b7d8-5a0acf92f880 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:38.616825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:38.617786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:38.618172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:38.618547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] PATH_INFO: `/v3/users/d583f50b5e5641db87e68d9f41388b26` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:38.620002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d583f50b5e5641db87e68d9f41388b26)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:38.622911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:38.683581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd583f50b5e5641db87e68d9f41388b26'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:38.684278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd583f50b5e5641db87e68d9f41388b26'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:38.698318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd583f50b5e5641db87e68d9f41388b26'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:38.708212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd583f50b5e5641db87e68d9f41388b26'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:38.716033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd583f50b5e5641db87e68d9f41388b26'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:38.723310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 927/1893] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:38 2022] DELETE /identity/v3/users/d583f50b5e5641db87e68d9f41388b26 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:38.728308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:38.728852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:38.728959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:38.751118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:38.752376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:38.819390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 967/1894] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:38.833094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:38.834748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:38.933641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8025d91-9518-42e2-8a7a-7ba140952b23 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:38.938061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:38.938748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:38.939059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:38.939522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:38.953717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:38.955193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:38.967294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 928/1895] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:39.395817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:39.398326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:39.415608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842a46e-47c6-4271-91af-51ac8a500356 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:39.428948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:39.430071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:39.430912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:39.431292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] PATH_INFO: `/v3/projects/e28a3b8520844694adf0244e458a3b8d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:39.432675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28a3b8520844694adf0244e458a3b8d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:39.439583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:39.544861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28a3b8520844694adf0244e458a3b8d', 'request_id': 'req-752fb150-6524-4aeb-974a-dd560867b0b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.555029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28a3b8520844694adf0244e458a3b8d', 'request_id': 'req-752fb150-6524-4aeb-974a-dd560867b0b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.578935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28a3b8520844694adf0244e458a3b8d', 'request_id': 'req-752fb150-6524-4aeb-974a-dd560867b0b8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.591832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] The token cache is being invalidate because project e28a3b8520844694adf0244e458a3b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:39.592166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.592851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.594367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 968/1896] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:39 2022] DELETE /identity/v3/projects/e28a3b8520844694adf0244e458a3b8d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:39.598293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:39.609289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419d493-7610-4067-9726-32a3ed6dd937 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:39.631497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:39.692059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:39.692711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:39.692995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:39.693273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] PATH_INFO: `/v3/users/a7f5a4a482204fdba3f5857fde7719f9` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:39.694682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7f5a4a482204fdba3f5857fde7719f9)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:39.696440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:39.756662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7f5a4a482204fdba3f5857fde7719f9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.757427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7f5a4a482204fdba3f5857fde7719f9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.769950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7f5a4a482204fdba3f5857fde7719f9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.779341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7f5a4a482204fdba3f5857fde7719f9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.787143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7f5a4a482204fdba3f5857fde7719f9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:39.794720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 929/1897] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:39 2022] DELETE /identity/v3/users/a7f5a4a482204fdba3f5857fde7719f9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:40.196104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:40.196728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:40.216807 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752fb150-6524-4aeb-974a-dd560867b0b8 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:40.228839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:40.231663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:40.343239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:40.345041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:40.345546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:40.346006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] PATH_INFO: `/v3/projects/8fb07a9c0c664f199fbe10323d3e33dd` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:40.347666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb07a9c0c664f199fbe10323d3e33dd)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:40.350119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:40.419763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:40.470645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb07a9c0c664f199fbe10323d3e33dd', 'request_id': 'req-302e8b46-f95b-4613-9084-a8448b71a969'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.482998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2adc17-3363-4945-9acb-883735ac0f63 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:40.489515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb07a9c0c664f199fbe10323d3e33dd', 'request_id': 'req-302e8b46-f95b-4613-9084-a8448b71a969'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.501522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb07a9c0c664f199fbe10323d3e33dd', 'request_id': 'req-302e8b46-f95b-4613-9084-a8448b71a969'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.520080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] The token cache is being invalidate because project 8fb07a9c0c664f199fbe10323d3e33dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:40.520522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.522111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.525164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 969/1898] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:40 2022] DELETE /identity/v3/projects/8fb07a9c0c664f199fbe10323d3e33dd => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:40.526561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:40.530104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:40.580905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:40.611784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:40.613355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:40.613745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:40.623026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] PATH_INFO: `/v3/users/b7ca3a7cb8604c4e9b22b01605e75b6b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:40.625451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ca3a7cb8604c4e9b22b01605e75b6b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:40.627609 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:40.667674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302e8b46-f95b-4613-9084-a8448b71a969 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:40.674237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:40.676172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:40.676903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:40.677579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] PATH_INFO: `/v3/users/11a0c98f7d4749228eefb12b1fa3c425` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:40.680910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=11a0c98f7d4749228eefb12b1fa3c425)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:40.683546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:40.777022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ca3a7cb8604c4e9b22b01605e75b6b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.778917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ca3a7cb8604c4e9b22b01605e75b6b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.796215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11a0c98f7d4749228eefb12b1fa3c425'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.797076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11a0c98f7d4749228eefb12b1fa3c425'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.800056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ca3a7cb8604c4e9b22b01605e75b6b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.811728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11a0c98f7d4749228eefb12b1fa3c425'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.821369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ca3a7cb8604c4e9b22b01605e75b6b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.838136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ca3a7cb8604c4e9b22b01605e75b6b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.860232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 930/1899] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:40 2022] DELETE /identity/v3/users/b7ca3a7cb8604c4e9b22b01605e75b6b => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:40.860232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:40.860500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11a0c98f7d4749228eefb12b1fa3c425'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.860585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:40.880184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fa3e5251-b853-4118-9a3b-431032e1d3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11a0c98f7d4749228eefb12b1fa3c425'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:40.906076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 970/1900] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:40 2022] DELETE /identity/v3/users/11a0c98f7d4749228eefb12b1fa3c425 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:40.914791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96df7-fd19-4f4a-b5a0-c503e2c1f2ab None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:40.914791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96df7-fd19-4f4a-b5a0-c503e2c1f2ab None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:40.916705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f96df7-fd19-4f4a-b5a0-c503e2c1f2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:40.988354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-16f96df7-fd19-4f4a-b5a0-c503e2c1f2ab None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:40.990013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-16f96df7-fd19-4f4a-b5a0-c503e2c1f2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.023679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751643e4-ce28-4a5b-85a1-e680b8da4323 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.067846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:41.139469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.140393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.140851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.141203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] PATH_INFO: `/v3/users/acb33ce183894caca6bfe6327e0309f3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.142690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb33ce183894caca6bfe6327e0309f3)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:41.142820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 971/1901] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:41.145379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.145750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.145998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:41.146126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.170234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:41.194814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:41.197624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.218598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 972/1902] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:41.219894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:41.222125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.245740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb33ce183894caca6bfe6327e0309f3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.247051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630b9f0-1e39-449f-96ce-7f326bb86d7b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.249466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb33ce183894caca6bfe6327e0309f3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.257272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.258310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.259513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.259884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] PATH_INFO: `/v3/users/6ce08d954c5f4962ac8e9639f3182c38` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.262836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ce08d954c5f4962ac8e9639f3182c38)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:41.265802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:41.288648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb33ce183894caca6bfe6327e0309f3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.302075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb33ce183894caca6bfe6327e0309f3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.319568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb33ce183894caca6bfe6327e0309f3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.343574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 931/1903] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:40 2022] DELETE /identity/v3/users/acb33ce183894caca6bfe6327e0309f3 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:41.343574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:41.347763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.409809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ce08d954c5f4962ac8e9639f3182c38'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.410969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ce08d954c5f4962ac8e9639f3182c38'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.427293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ce08d954c5f4962ac8e9639f3182c38'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.479667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ce08d954c5f4962ac8e9639f3182c38'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.491398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ce08d954c5f4962ac8e9639f3182c38'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:41.508052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 973/1904] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:41 2022] DELETE /identity/v3/users/6ce08d954c5f4962ac8e9639f3182c38 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:41.516200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:41.519378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.527903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235e84f-2cf9-4484-88f5-4d58bd1faab1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.712122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d4956-b208-4534-bd61-2a3d36e8bb28 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.728674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.729237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.729548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.729836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.732818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.733466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.733755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.734032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.734381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.742587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.836066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:41.838921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:41.850560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:41.856636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 932/1905] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:41.860347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:41.867261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:41.871899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e2041-31eb-4663-b54d-45942110dcce service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.880218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.881039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:41.881464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.881839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.882486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.884923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd96ce-a37f-4bd6-9a7e-e37f0f89bd83 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:41.885472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.898590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 974/1906] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:41.899841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.900135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.900411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:41.901213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:41.902964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:41.915475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 933/1907] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:41.921594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:41.923308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.934881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 975/1908] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:41.956257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:41.958414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:41.979275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f921c2-627f-412a-8100-a9078b5636bd service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:41.992481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:41.993581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:41.993921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:41.994255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.009936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:42.011709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de7f51-47e9-42d1-9795-44740d34fdda service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:42.025028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 934/1909] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:42.141164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:42.143403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:42.171315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03bb5e-60d9-4c93-8353-f54647d01db5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:42.182492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:42.183920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:42.184544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:42.184973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] PATH_INFO: `/v3/projects/06ddd06f4fb847c0b6dedbef26d27321` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.203033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ddd06f4fb847c0b6dedbef26d27321)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:42.213273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:42.417158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ddd06f4fb847c0b6dedbef26d27321', 'request_id': 'req-918a2a85-390c-475d-a741-0a7b6d79c529'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.469834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ddd06f4fb847c0b6dedbef26d27321', 'request_id': 'req-918a2a85-390c-475d-a741-0a7b6d79c529'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.489359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ddd06f4fb847c0b6dedbef26d27321', 'request_id': 'req-918a2a85-390c-475d-a741-0a7b6d79c529'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.527773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] The token cache is being invalidate because project 06ddd06f4fb847c0b6dedbef26d27321 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:42.528700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.530273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.533632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 976/1910] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:42 2022] DELETE /identity/v3/projects/06ddd06f4fb847c0b6dedbef26d27321 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:42.544410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:42.545192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:42.545748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.573842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:42.576376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:42.582057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:42.587649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:42.706939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 935/1911] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:42.706939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:42.706939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:42.735673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82668d40-09f3-41d6-852b-d8b688fa03b2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:42.757456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:42.758100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:42.760602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:42.760935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] PATH_INFO: `/v3/projects/09a21841ec1344798ac58f5f1d709641` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.765149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a21841ec1344798ac58f5f1d709641)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:42.782165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:42.782280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a2a85-390c-475d-a741-0a7b6d79c529 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:42.792132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:42.795009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:42.796144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:42.797388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] PATH_INFO: `/v3/projects/1b6ac29e18f94f81883f8f70d674d737` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.800667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b6ac29e18f94f81883f8f70d674d737)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:42.804980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:42.855469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a21841ec1344798ac58f5f1d709641', 'request_id': 'req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.868547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a21841ec1344798ac58f5f1d709641', 'request_id': 'req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.887397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a21841ec1344798ac58f5f1d709641', 'request_id': 'req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.888155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b6ac29e18f94f81883f8f70d674d737', 'request_id': 'req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.897973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b6ac29e18f94f81883f8f70d674d737', 'request_id': 'req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.900741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] The token cache is being invalidate because project 09a21841ec1344798ac58f5f1d709641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:42.901164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.902678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.904087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 936/1912] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:42 2022] DELETE /identity/v3/projects/09a21841ec1344798ac58f5f1d709641 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:42.905050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:42.906550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:42.908375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b6ac29e18f94f81883f8f70d674d737', 'request_id': 'req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.916599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:42.924815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] The token cache is being invalidate because project 1b6ac29e18f94f81883f8f70d674d737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:42.928053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.929480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-16b8d5cc-bced-4dce-b5e6-295a5e8fdb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:42.932781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 977/1913] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:42 2022] DELETE /identity/v3/projects/1b6ac29e18f94f81883f8f70d674d737 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:42.934470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:42.934773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:42.935009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:42.972894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:42.974372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.000864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b82b-57dc-4a18-873d-9c9be2fb9cda admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.007089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:43.090684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.091583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.091839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.092124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] PATH_INFO: `/v3/projects/2cc5dd59014b49c39d23c1b613c818e2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.094914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc5dd59014b49c39d23c1b613c818e2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.096215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 978/1914] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:43.098286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.099100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.104705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.118608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f20cb-8466-424b-960b-89830c849399 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.123652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.124335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.124673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.124956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] PATH_INFO: `/v3/users/abd13157414546f085f2558514f25ae6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.126099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd13157414546f085f2558514f25ae6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.128139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.182891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc5dd59014b49c39d23c1b613c818e2', 'request_id': 'req-b3b7d3b3-2348-418d-bad5-c819d2141fd7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.188581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd13157414546f085f2558514f25ae6'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.189423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd13157414546f085f2558514f25ae6'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.197144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc5dd59014b49c39d23c1b613c818e2', 'request_id': 'req-b3b7d3b3-2348-418d-bad5-c819d2141fd7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.204020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd13157414546f085f2558514f25ae6'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.204515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc5dd59014b49c39d23c1b613c818e2', 'request_id': 'req-b3b7d3b3-2348-418d-bad5-c819d2141fd7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.218257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] The token cache is being invalidate because project 2cc5dd59014b49c39d23c1b613c818e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:43.218623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.220057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.221932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 937/1915] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:42 2022] DELETE /identity/v3/projects/2cc5dd59014b49c39d23c1b613c818e2 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:43.222286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd13157414546f085f2558514f25ae6'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.226086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.227841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.232693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd13157414546f085f2558514f25ae6'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.241686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 979/1916] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:43 2022] DELETE /identity/v3/users/abd13157414546f085f2558514f25ae6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:43.243115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.245320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.434558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b7d3b3-2348-418d-bad5-c819d2141fd7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.442090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.442918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.443089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.443470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] PATH_INFO: `/v3/users/fd0c9d872c4e47f9b84eef89460c2472` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.445141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd0c9d872c4e47f9b84eef89460c2472)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.446054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbc0cc-912a-4707-b3f8-0e9a9e2ad856 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.448558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.462532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.463457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.464038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.464456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.465453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:43.465909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.468191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.490369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 980/1917] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:43.492117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.495034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.513396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:43.522345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44602850-804f-4406-b236-8ffabbca6905 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.540835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:43.551590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd0c9d872c4e47f9b84eef89460c2472'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.552454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd0c9d872c4e47f9b84eef89460c2472'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.556804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.558269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.558782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.559362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] PATH_INFO: `/v3/users/e2d6ed8dcc4d469ea4b59abd5a8569cc` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.569421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2d6ed8dcc4d469ea4b59abd5a8569cc)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.575439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd0c9d872c4e47f9b84eef89460c2472'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.576123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.597595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd0c9d872c4e47f9b84eef89460c2472'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.621915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd0c9d872c4e47f9b84eef89460c2472'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.631285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 938/1918] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:43 2022] DELETE /identity/v3/users/fd0c9d872c4e47f9b84eef89460c2472 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:43.632056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.634427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.681631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2d6ed8dcc4d469ea4b59abd5a8569cc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.682876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2d6ed8dcc4d469ea4b59abd5a8569cc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.695521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2d6ed8dcc4d469ea4b59abd5a8569cc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.722882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2d6ed8dcc4d469ea4b59abd5a8569cc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.733538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2d6ed8dcc4d469ea4b59abd5a8569cc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:43.745838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 981/1919] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:43 2022] DELETE /identity/v3/users/e2d6ed8dcc4d469ea4b59abd5a8569cc => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:43.757206 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92118dc6-2302-4a5e-9ddf-aec4e093a57e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.767081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:43.770994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:43.926947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ec602-b873-47e5-8896-2d345abfae7a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:43.934191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.935222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.935579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.935901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] PATH_INFO: `/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.937259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8d2fda3dd54e6f85fc85bd2401643c)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.940095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:43.940822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:43.943022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:43.944063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:43.945102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:43.947038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:43.952178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.008975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 939/1920] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:43 2022] POST /identity/v3/projects => generated 373 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:44.012991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.015401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.026853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8d2fda3dd54e6f85fc85bd2401643c', 'request_id': 'req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.039112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8d2fda3dd54e6f85fc85bd2401643c', 'request_id': 'req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.049292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8d2fda3dd54e6f85fc85bd2401643c', 'request_id': 'req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.073872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] The token cache is being invalidate because project 0f8d2fda3dd54e6f85fc85bd2401643c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:44.073872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.074303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2ba4d0b4-a39c-4f76-b5e5-e1e598a05eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.083872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 982/1921] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:43 2022] DELETE /identity/v3/projects/0f8d2fda3dd54e6f85fc85bd2401643c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:44.088584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63f06-109f-4b47-9e40-8db80dfc9012 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.088584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63f06-109f-4b47-9e40-8db80dfc9012 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.088584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa63f06-109f-4b47-9e40-8db80dfc9012 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.093674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 983/1922] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:00:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:00:44.093674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab74e-5584-4297-b7ed-04fc5917011c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.093674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab74e-5584-4297-b7ed-04fc5917011c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.093674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ab74e-5584-4297-b7ed-04fc5917011c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.108076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:44.133251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-324ab74e-5584-4297-b7ed-04fc5917011c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:44.142105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:44.152547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-324ab74e-5584-4297-b7ed-04fc5917011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.175932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16893b83-62a6-4bb6-bb9e-347b5e977162 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.232090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 984/1923] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:44.233412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.233701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.234096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.255731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.256279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.256561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.256834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.257473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.259495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db03dd3-a077-45e2-a197-acb8a8d775a8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.271489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:44.273924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.286717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 940/1924] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/users => generated 287 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:44.288368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.288714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.289098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.320568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:44.321890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.342410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 941/1925] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:44.345761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.349066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.390635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b8a6c-8152-4c75-9189-a04f0e8a7539 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.398874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.400495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.401520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.401946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.403421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.407066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.419117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 985/1926] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:44.420023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.421396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.430818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 942/1927] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:44.432459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.443048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626de83-fc63-4a2f-b614-4d478dbf3987 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.449634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd172b81-6f7b-48f5-9ed7-df011f6b5c09 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.456836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.457558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.457655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.457846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.458169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] PATH_INFO: `/v3/projects/1088d03104924c78834aff933f86a686` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.458638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.459250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.459412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1088d03104924c78834aff933f86a686)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.459725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.460873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:44.461399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.461818 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.464290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.485833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 943/1928] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:44.487245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.490605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.514399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64392477-daed-4d80-8ceb-b0957c7e9d85 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.521310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1088d03104924c78834aff933f86a686', 'request_id': 'req-2b8d304e-3228-4394-80fa-59abe68c9a0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.530807 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1088d03104924c78834aff933f86a686', 'request_id': 'req-2b8d304e-3228-4394-80fa-59abe68c9a0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.538999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1088d03104924c78834aff933f86a686', 'request_id': 'req-2b8d304e-3228-4394-80fa-59abe68c9a0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.556528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] The token cache is being invalidate because project 1088d03104924c78834aff933f86a686 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:44.557159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.558507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.560871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 986/1929] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:44 2022] DELETE /identity/v3/projects/1088d03104924c78834aff933f86a686 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:44.565997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.565997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.604852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:44.615101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.616587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.617023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.617180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] PATH_INFO: `/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.618885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aedf6a9259f4f05a93e891e4aca1f34)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.620891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.674804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d304e-3228-4394-80fa-59abe68c9a0f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.684936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aedf6a9259f4f05a93e891e4aca1f34', 'request_id': 'req-525a3b5e-d47b-4561-ac9f-44837a868523'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.711322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aedf6a9259f4f05a93e891e4aca1f34', 'request_id': 'req-525a3b5e-d47b-4561-ac9f-44837a868523'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.721323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:44.741048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aedf6a9259f4f05a93e891e4aca1f34', 'request_id': 'req-525a3b5e-d47b-4561-ac9f-44837a868523'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.776146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] The token cache is being invalidate because project 2aedf6a9259f4f05a93e891e4aca1f34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:44.777932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.781096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:44.789308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 944/1930] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:44 2022] DELETE /identity/v3/projects/2aedf6a9259f4f05a93e891e4aca1f34 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:44.789308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.789853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.832673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.833309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.833644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.833931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.835042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.837517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.852543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 987/1931] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:44.853308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:44.855078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:44.869955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058aa4a6-0d92-48d0-ad5d-a6cd8d459adf admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.875982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:44.876664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.887287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.887888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.890218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:44.896096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c485dad0-06b0-4902-8dcd-98f563ccd694 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:44.936024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 988/1932] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:44.936284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a3b5e-d47b-4561-ac9f-44837a868523 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:44.936394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c669c-8900-4391-9f6a-2beebebcfcac None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:44.936394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c669c-8900-4391-9f6a-2beebebcfcac None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:44.936394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c669c-8900-4391-9f6a-2beebebcfcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:44.955492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-068c669c-8900-4391-9f6a-2beebebcfcac None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:44.957468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-068c669c-8900-4391-9f6a-2beebebcfcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.020674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.021479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.021864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.022230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.024269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.043828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 989/1933] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.044019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.044107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.044107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.044107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.044365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.060114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 945/1934] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:45.061639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.063621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.070800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:45.072813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.085783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 990/1935] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.086114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1bbed-316c-426f-a71a-0d00c7adb632 service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.093093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.093870 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.094994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.095558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.096377 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] PATH_INFO: `/v3/projects/59c9cf58e85a49b2a8db7c7c29367ada/users/9000dd577a344dde85a882f2a2d51ea1/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.096500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.118013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40498ea9-5dd7-44b5-b4ad-bdb0ab9b0045 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.123095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c9cf58e85a49b2a8db7c7c29367ada, user_id=9000dd577a344dde85a882f2a2d51ea1, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.125812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.126448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.127683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.129615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.129615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] PATH_INFO: `/v3/users/1c02d99a16d642d1a3bf015459e9d483` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.131841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c02d99a16d642d1a3bf015459e9d483)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.135470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.142516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 946/1936] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:45 2022] PUT /identity/v3/projects/59c9cf58e85a49b2a8db7c7c29367ada/users/9000dd577a344dde85a882f2a2d51ea1/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:45.143688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.210559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:45.217283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:45.245299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad8767-abec-427f-923e-b689816f9792 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.246787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c02d99a16d642d1a3bf015459e9d483'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:45.250654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.251275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.251554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.251863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.252384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c02d99a16d642d1a3bf015459e9d483'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:45.252498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.255072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1bba0-d85b-4a6a-8d01-5471df0a0547 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.266206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c02d99a16d642d1a3bf015459e9d483'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:45.282762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 947/1937] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/users => generated 289 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:45.287220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43953-a4bc-41b8-b4d1-a73b7afc9a9c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.287503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43953-a4bc-41b8-b4d1-a73b7afc9a9c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.287822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c43953-a4bc-41b8-b4d1-a73b7afc9a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.296055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c02d99a16d642d1a3bf015459e9d483'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:45.308701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a3e180b-a5e5-4c37-b8b4-d20c5b932a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c02d99a16d642d1a3bf015459e9d483'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:45.311371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c4c43953-a4bc-41b8-b4d1-a73b7afc9a9c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:45.313211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c4c43953-a4bc-41b8-b4d1-a73b7afc9a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.317209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 991/1938] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:45 2022] DELETE /identity/v3/users/1c02d99a16d642d1a3bf015459e9d483 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:45.318676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.319093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.319488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.345972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:45.347917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.425944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 948/1939] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.428256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.428761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.429284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.431121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 992/1940] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.432559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.435378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.448564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be220c21-fc48-4ac9-98b6-710e7daa9255 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.453602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.454355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.454729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.455098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.456073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.461027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49b2e18-8f0c-46dc-865c-0616b406cd45 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.470643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 993/1941] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:45.472216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.472636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.473108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.481886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] MFA Rules not processed for user `9000dd577a344dde85a882f2a2d51ea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:45.483245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.493078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:45.494401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.502656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 994/1942] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.504119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.514240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee20439-8336-4a25-bc51-3d3c6ef82ba0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.519504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.520320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.520667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.521014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.521736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:45.522110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.523989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.541474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 995/1943] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:45.541474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.552692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa4eaa-5e1d-44a8-a867-1dba00a10b4e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.558748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.559684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.559684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.560054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.560596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:45.560964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.562860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.578578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 996/1944] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:45.579410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.597819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 949/1945] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:45.598653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc8896-34c7-4d6f-bc8f-f4920093b3d0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.599076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.600791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.609487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.609487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.609902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.610409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.611119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.613553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.626569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 997/1946] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:45.627212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.641675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be28e75-66c9-41a4-b44a-4e31c09363e1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.654822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.655533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.655937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.656310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.657047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.659120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.698524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 998/1947] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:45.699980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.718132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a73e1-9e56-41a8-824d-45d9db0f020e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.741452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:45.758443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:45.794900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.795829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.796334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.796823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.797930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ad19f-d188-4117-b03b-21c51b0c4a86 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.806203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.807094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.807379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.807652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.807652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.811403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.812858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.815221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:45.832248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 950/1948] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:45.833836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.836204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:45.866385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 999/1949] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:45.867333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:45.870627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f3ede-90db-4c97-8dff-eeef72f0b0c4 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:45.972450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:45.973694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:45.973899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:45.974186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:45.974981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:45.978580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.004846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1217ef-e55b-44aa-8236-e44d0d9f9dad admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.011632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.015320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.015320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.015320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] PATH_INFO: `/v3/projects/d321dbfe7d6c4600b6b1d40d1e55c655/users/b63b0236f8584b9fbdee6f0e52ad64f5/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.049394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d321dbfe7d6c4600b6b1d40d1e55c655, user_id=b63b0236f8584b9fbdee6f0e52ad64f5, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.052563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 951/1950] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:45 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:46.053560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.053999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.078483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1000/1951] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:45 2022] PUT /identity/v3/projects/d321dbfe7d6c4600b6b1d40d1e55c655/users/b63b0236f8584b9fbdee6f0e52ad64f5/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:46.079686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.199504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655c4ab-a83a-4de3-8cfc-2810c29630f9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.260687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8feb36-87a5-449e-9621-0909c1138d64 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.270071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.271055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.271530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.272048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.272166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:46.273309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.278877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bd88cf-386c-46de-89f2-acad4b75e6cc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.281640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.282640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.284866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.285753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.286728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.289017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.322374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 952/1952] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:46 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:46.323880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.328484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1001/1953] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:46 2022] POST /identity/v3/users => generated 285 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:46.330520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.331019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.331736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.339191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2539fe-f89e-45fc-8c7c-d171831d0d8a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.346937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.348026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.348379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.348720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.349473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.351494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.380239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] MFA Rules not processed for user `b63b0236f8584b9fbdee6f0e52ad64f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:46.382890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.391355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 953/1954] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:46 2022] POST /identity/v3/users => generated 294 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:46.395957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.413023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7e98f-8f87-4425-a7bf-0c5bee5c3f66 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.425910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:46.430229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.434267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.434558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.434857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.436382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.438834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.457248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 954/1955] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:46.458221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.480463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f84de-9ee8-439c-a3d8-3b44d6248747 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.487102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.488177 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.488875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.489158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.489917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.493231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.507971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 955/1956] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:46.509094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.511436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.529097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90055f31-79c4-4a8c-80b0-d6405dd2745b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.535545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.536523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.536964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.537387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] PATH_INFO: `/v3/projects/cda1ec7bd11343aa94ee32bfbce01fea` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.540010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda1ec7bd11343aa94ee32bfbce01fea)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.542594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.550675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1002/1957] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:46.555641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.569548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e03d2-0653-4df4-af07-e56385bb2a43 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.575783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.576549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.576929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.577994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.578764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.580826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.611849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1003/1958] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:46.611849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.618126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda1ec7bd11343aa94ee32bfbce01fea', 'request_id': 'req-9b672c74-7451-4ecb-8425-d812a4156ca4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.620882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3412f-4e99-4cc2-a513-4d8528e79fc1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.628133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda1ec7bd11343aa94ee32bfbce01fea', 'request_id': 'req-9b672c74-7451-4ecb-8425-d812a4156ca4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.637859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda1ec7bd11343aa94ee32bfbce01fea', 'request_id': 'req-9b672c74-7451-4ecb-8425-d812a4156ca4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.650138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] The token cache is being invalidate because project cda1ec7bd11343aa94ee32bfbce01fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:46.650421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.651140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.653378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 956/1959] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:46 2022] DELETE /identity/v3/projects/cda1ec7bd11343aa94ee32bfbce01fea => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:46.656655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.657742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.695457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.696386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.696804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.697210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.698188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.700871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.721116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1004/1960] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:46.721116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.721577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.725249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672c74-7451-4ecb-8425-d812a4156ca4 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.729914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.730530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.730812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.731088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.731739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.733493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.741200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970b7ff-3888-4298-b7db-de06f6a1ca4e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.745158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 957/1961] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:46.746031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.748009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.748446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.749282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.749701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.750108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.751096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.754936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.762080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb5de9-146a-4e68-b72c-01ac3f0805ac admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.767261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.767979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.768263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.768631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] PATH_INFO: `/v3/projects/b01760e2d68240d9aeccfb440fd40d8b/users/f6c23810b76645079487f2757976851a/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.784331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01760e2d68240d9aeccfb440fd40d8b, user_id=f6c23810b76645079487f2757976851a, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.787220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.796176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1005/1962] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:46 2022] POST /identity/v3/projects => generated 377 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:46.797561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.801179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.801793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 958/1963] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:46 2022] PUT /identity/v3/projects/b01760e2d68240d9aeccfb440fd40d8b/users/f6c23810b76645079487f2757976851a/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:46.802600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.804262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:46.809308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:46.875496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ceaa68-e31a-4b1f-a2a3-396981e35198 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.880772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.881459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.881736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.882011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] PATH_INFO: `/v3/projects/9808329d33aa42d29c3b4ad55cd7761e/users/e4d8ddfc15ad46718d93eda0cb90a978/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.889821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cabcb-aaf0-49e7-bace-40fd62f2340d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:46.895954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:46.896806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:46.897180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:46.897512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] PATH_INFO: `/v3/users/f12353c394264ba2a1ff67a4eedaae8c` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:46.898952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12353c394264ba2a1ff67a4eedaae8c)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.902530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.903274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9808329d33aa42d29c3b4ad55cd7761e, user_id=e4d8ddfc15ad46718d93eda0cb90a978, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:46.907465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:46.921223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 959/1964] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:46 2022] PUT /identity/v3/projects/9808329d33aa42d29c3b4ad55cd7761e/users/e4d8ddfc15ad46718d93eda0cb90a978/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:46.922035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:46.936522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:46.967455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12353c394264ba2a1ff67a4eedaae8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.968185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12353c394264ba2a1ff67a4eedaae8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.977697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12353c394264ba2a1ff67a4eedaae8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:46.993539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12353c394264ba2a1ff67a4eedaae8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.004817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12353c394264ba2a1ff67a4eedaae8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.013327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4adbca-eeee-4854-be85-c1adc6876fb1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.013902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1006/1965] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:46 2022] DELETE /identity/v3/users/f12353c394264ba2a1ff67a4eedaae8c => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:47.015355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.017146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.137778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469dca4-e0f5-4e68-b238-493c3ac2ae32 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.146372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.147409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.147856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.148279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] PATH_INFO: `/v3/projects/ba7a6d7160f24e589f1180696dccb60a/users/1abd7907b483440883321b1249b49dcb/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.154580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.155365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.156939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.157327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.158120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.160461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.176007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7a6d7160f24e589f1180696dccb60a, user_id=1abd7907b483440883321b1249b49dcb, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.181099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630952e-1b85-4d29-b022-4f851f02dad0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.210847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1007/1966] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:47 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:47.212299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.212530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 960/1967] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:46 2022] PUT /identity/v3/projects/ba7a6d7160f24e589f1180696dccb60a/users/1abd7907b483440883321b1249b49dcb/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:47.217919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.218359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.218780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.221927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.237382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea827c09-1d47-42c1-98d1-7fce21b92add admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.277608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] MFA Rules not processed for user `e4d8ddfc15ad46718d93eda0cb90a978`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:47.279406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.286124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:47.313167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.313987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.314122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.314837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.315057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.317593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.333447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1008/1968] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:47.333447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.340691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723452c-9c41-4c7d-b8c8-f68bc37b3c00 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.346756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.347465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.347782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.347984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.349153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.350787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56208f13-dd0b-4e51-bbcc-46ee741554c5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.363836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1009/1969] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:47.365024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.365314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.365590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.396353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] MFA Rules not processed for user `1abd7907b483440883321b1249b49dcb`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:47.397770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.402121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:47.422140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 961/1970] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:47.422140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.440699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38db01-e891-4cbb-a98e-88436eae4c6a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.447663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.448376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.448655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.448956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] PATH_INFO: `/v3/projects/b01760e2d68240d9aeccfb440fd40d8b/users/f6c23810b76645079487f2757976851a/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.451091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01760e2d68240d9aeccfb440fd40d8b, user_id=f6c23810b76645079487f2757976851a, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.453991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.471248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 962/1971] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:47 2022] PUT /identity/v3/projects/b01760e2d68240d9aeccfb440fd40d8b/users/f6c23810b76645079487f2757976851a/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:47.472400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.531279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1010/1972] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:47.537760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.540420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.584105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136db5cc-bd09-44bf-bf5e-68554905acba admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.597243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.598294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.599022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.599339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] PATH_INFO: `/v3/projects/b0997b80ba0047d48fb86f1ffbf82645/users/b73f4f37c85d49219dc686e23a7da1d9/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.620360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0997b80ba0047d48fb86f1ffbf82645, user_id=b73f4f37c85d49219dc686e23a7da1d9, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.622271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552bfb8-8a23-4abf-8f30-a2a83e2fd9ae None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.623483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.650549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 963/1973] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:47 2022] PUT /identity/v3/projects/b0997b80ba0047d48fb86f1ffbf82645/users/b73f4f37c85d49219dc686e23a7da1d9/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:47.655219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.657689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.730931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.731832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.732123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.732397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.741537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a0cd4-e53b-4d20-a411-87e06a379471 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.746829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.747453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.747737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.748070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] PATH_INFO: `/v3/projects/428dd007ad6d456ab255695f3cbe1f0e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.749368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=428dd007ad6d456ab255695f3cbe1f0e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.751360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.770526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:47.814293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '428dd007ad6d456ab255695f3cbe1f0e', 'request_id': 'req-de90bac2-fe5a-4c27-b094-48e1e1286ed0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.815842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.818767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b940d-1cc4-4ea4-a044-34b981e9ba30 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:47.825028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '428dd007ad6d456ab255695f3cbe1f0e', 'request_id': 'req-de90bac2-fe5a-4c27-b094-48e1e1286ed0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.832920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '428dd007ad6d456ab255695f3cbe1f0e', 'request_id': 'req-de90bac2-fe5a-4c27-b094-48e1e1286ed0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.833423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1011/1974] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:47.834888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.835310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.835585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.868113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] The token cache is being invalidate because project 428dd007ad6d456ab255695f3cbe1f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:47.868469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.869119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:47.871869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 964/1975] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:47 2022] DELETE /identity/v3/projects/428dd007ad6d456ab255695f3cbe1f0e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:47.872298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:47.873780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.881056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] MFA Rules not processed for user `f6c23810b76645079487f2757976851a`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:47.882697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:47.888075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:47.979166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90bac2-fe5a-4c27-b094-48e1e1286ed0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:47.986560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:47.987312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:47.987645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:47.989123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:47.990044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:47.992372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.009998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1012/1976] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:48.011500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.013326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.028907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c0837-521c-4dfa-9ad2-21e84a9a666a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.036906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.037793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.038033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 965/1977] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:47 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:48.038196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.038965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.041227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.041936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.042334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.043650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.055831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1013/1978] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:48.056793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.058519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.073006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd83f5a0-c182-4e04-91f7-a6703a93ea3f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.136101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69d33f-d2be-46b9-978c-9e9f20057114 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.148554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.150435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.150974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.151380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] PATH_INFO: `/v3/users/e4d8ddfc15ad46718d93eda0cb90a978` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.153329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d8ddfc15ad46718d93eda0cb90a978)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.155894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.167284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.167964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.168249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.168566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] PATH_INFO: `/v3/users/6cc5ffc217554185983a4918b166dce0` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.170048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc5ffc217554185983a4918b166dce0)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.172246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.225752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:48.246781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc5ffc217554185983a4918b166dce0'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.247365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d8ddfc15ad46718d93eda0cb90a978'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.247508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc5ffc217554185983a4918b166dce0'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.248087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d8ddfc15ad46718d93eda0cb90a978'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.256548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc5ffc217554185983a4918b166dce0'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.262555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d8ddfc15ad46718d93eda0cb90a978'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.274618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d8ddfc15ad46718d93eda0cb90a978'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.278802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc5ffc217554185983a4918b166dce0'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.293037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d8ddfc15ad46718d93eda0cb90a978'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.304098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 966/1979] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:48 2022] DELETE /identity/v3/users/e4d8ddfc15ad46718d93eda0cb90a978 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:48.305132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.305758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc5ffc217554185983a4918b166dce0'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:48.307249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.339390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1014/1980] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:48 2022] DELETE /identity/v3/users/6cc5ffc217554185983a4918b166dce0 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:48.341837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.360612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.383409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:48.390967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99793f6-d618-43cb-994b-9ec93e5a63b5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.481686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.482669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.483194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.483676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.487203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.493172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd77066-17ac-4bee-92f2-42db983f14bd admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.506674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.507589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.508461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.508875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.510448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.514378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.562963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1015/1981] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:48 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:48.564268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.566216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.580747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548935e-487c-4e20-880a-fb19e35ec905 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.587104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.588282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.588564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.589356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] PATH_INFO: `/v3/projects/b0997b80ba0047d48fb86f1ffbf82645/users/b73f4f37c85d49219dc686e23a7da1d9/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.591155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0997b80ba0047d48fb86f1ffbf82645, user_id=b73f4f37c85d49219dc686e23a7da1d9, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.594129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ff5f0-0ac9-4c71-83f3-92495c95964c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.606995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1016/1982] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:48 2022] PUT /identity/v3/projects/b0997b80ba0047d48fb86f1ffbf82645/users/b73f4f37c85d49219dc686e23a7da1d9/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:48.617648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593c45a-0e35-4e9d-97b8-bf17d6f3da84 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.617850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593c45a-0e35-4e9d-97b8-bf17d6f3da84 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.618130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593c45a-0e35-4e9d-97b8-bf17d6f3da84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.627922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.632413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.640045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0593c45a-0e35-4e9d-97b8-bf17d6f3da84 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:48.641601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0593c45a-0e35-4e9d-97b8-bf17d6f3da84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.653442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 967/1983] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:48.655184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.657893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.686034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fab386-03a1-46af-817c-af81fb780868 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.718545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:48.750295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1017/1984] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:48.750295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.750295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.750295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.786045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] MFA Rules not processed for user `b73f4f37c85d49219dc686e23a7da1d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:48.789868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.805742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.807427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.807933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.808403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.809495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.813804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.843988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 968/1985] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:48.845950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.850448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:48.908169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1018/1986] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:48.909959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.939392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9694929e-9eb9-4c01-ade2-a94a7125e507 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.946917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d6f8-a119-4c4a-be34-0068cc01ba99 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:48.949118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.950264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.950675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.951117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] PATH_INFO: `/v3/projects/1ba839a1f21f437c8f7125591f7ff8be/users/bbccd5c09a294e11b3b1a32bc53c169d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.952002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:48.952765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:48.953288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:48.954789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:48.971059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:48.973846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:48.980886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:48.987499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 969/1987] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:48.989213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:48.993349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.010481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba839a1f21f437c8f7125591f7ff8be, user_id=bbccd5c09a294e11b3b1a32bc53c169d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.011147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.067835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1019/1988] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:48 2022] PUT /identity/v3/projects/1ba839a1f21f437c8f7125591f7ff8be/users/bbccd5c09a294e11b3b1a32bc53c169d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:49.069377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.073521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.118143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc02de-5c83-4ffa-9908-451d5e888eb3 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.199964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b16b51-8ffc-4e45-8fe2-9e7e120b08f0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.206539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.207496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.207937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.208217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] PATH_INFO: `/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.209373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a22a513a45f9453da7a7d9d8de8ad2db)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.211690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.212540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.213182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.213469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.213799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.215693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.268627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:49.297435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a22a513a45f9453da7a7d9d8de8ad2db', 'request_id': 'req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.318984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a22a513a45f9453da7a7d9d8de8ad2db', 'request_id': 'req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.331193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a22a513a45f9453da7a7d9d8de8ad2db', 'request_id': 'req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.332818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.335152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.349645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 970/1989] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:49.351035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.360862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] The token cache is being invalidate because project a22a513a45f9453da7a7d9d8de8ad2db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:49.361324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.361825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-68e2e516-7dd7-4acd-9c8c-97f73b0dbe99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.368694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1020/1990] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:49 2022] DELETE /identity/v3/projects/a22a513a45f9453da7a7d9d8de8ad2db => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:49.370651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a5d7b-67bb-4938-baf0-93a68d43616f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.370651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a5d7b-67bb-4938-baf0-93a68d43616f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.370916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a5d7b-67bb-4938-baf0-93a68d43616f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.376464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6c71c-eca2-491b-9b51-aaaab6ef4325 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.396987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-246a5d7b-67bb-4938-baf0-93a68d43616f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:49.398323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-246a5d7b-67bb-4938-baf0-93a68d43616f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.567625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1021/1991] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:49.581240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.583936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.585879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.593710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.595291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.596869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.598014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.604141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.617774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.636756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:49.638905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.653934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1022/1992] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:49.653934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.653934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.663956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 971/1993] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:49 2022] GET /identity/v3/roles => generated 1420 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:49.664804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.670379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41620-6af1-4467-a50d-6fb2ed7f7eb0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.686715 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.689496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.689764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.690190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] PATH_INFO: `/v3/projects/9808329d33aa42d29c3b4ad55cd7761e` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.692145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9808329d33aa42d29c3b4ad55cd7761e)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.695382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.703093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308f09e-4556-4097-a0c2-67586773ee4a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.706696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:49.718441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.721473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.722090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.723516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.724945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:49.725499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.729396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.757707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 972/1994] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:49.759075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.763507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.798833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aad133-5309-44e9-b895-0d587b58d81f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.806149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9808329d33aa42d29c3b4ad55cd7761e', 'request_id': 'req-94cff07c-7b86-4627-a632-520a8970ce28'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.815413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9808329d33aa42d29c3b4ad55cd7761e', 'request_id': 'req-94cff07c-7b86-4627-a632-520a8970ce28'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.823484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9808329d33aa42d29c3b4ad55cd7761e', 'request_id': 'req-94cff07c-7b86-4627-a632-520a8970ce28'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.836148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] The token cache is being invalidate because project 9808329d33aa42d29c3b4ad55cd7761e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:49.836444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.837128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:49.839008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1023/1995] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:49 2022] DELETE /identity/v3/projects/9808329d33aa42d29c3b4ad55cd7761e => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:49.839782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:49.841329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:49.962865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cff07c-7b86-4627-a632-520a8970ce28 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:49.971495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.973026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.975660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.976089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:49.976869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:49.979166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4811adf-13e7-452c-bdfd-885384e18037 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:49.996995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:49.997781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:49.997987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:49.998440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] PATH_INFO: `/v3/projects/1ba839a1f21f437c8f7125591f7ff8be/users/bbccd5c09a294e11b3b1a32bc53c169d/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.000865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ba839a1f21f437c8f7125591f7ff8be, user_id=bbccd5c09a294e11b3b1a32bc53c169d, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.004104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.015968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1024/1996] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:49 2022] POST /identity/v3/projects => generated 389 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:50.022387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197a38-4d94-4a0c-828f-8d058f1ac020 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.022951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197a38-4d94-4a0c-828f-8d058f1ac020 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.023318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197a38-4d94-4a0c-828f-8d058f1ac020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.034048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 973/1997] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:49 2022] PUT /identity/v3/projects/1ba839a1f21f437c8f7125591f7ff8be/users/bbccd5c09a294e11b3b1a32bc53c169d/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:50.037131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.058862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7c197a38-4d94-4a0c-828f-8d058f1ac020 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:50.060533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7c197a38-4d94-4a0c-828f-8d058f1ac020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.142940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1025/1998] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:50.145327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.145327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.145814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.182633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:50.184576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] MFA Rules not processed for user `bbccd5c09a294e11b3b1a32bc53c169d`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:50.186418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.199840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:50.230754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f78920a-0431-46fa-b4e8-4cf28ed09d7d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.238558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.239264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.239621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.239957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.240918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.243180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784fc16-c405-4f2d-8967-e9bfefbcb8d9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.290061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 974/1999] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/users => generated 295 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:50.291497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.292021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.292878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.319209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:50.320663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.327847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1026/2000] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:50.327847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.337667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 975/2001] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:50.340121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5bba1-8a4f-4d98-8bda-2945a4b5886a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.344437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.347753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.348692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.349206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.349628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.350529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.352391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.361934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c56a43-9404-4cff-b946-de18e23d67eb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.369588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.370693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.371141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.371599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.372449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:50.372896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.375186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.377704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1027/2002] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:50.382748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.399607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfe70b-cb77-44bb-81b4-3305658aff02 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.407832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 976/2003] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:50.408088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.408088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.408088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.408088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.408953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.409447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.411402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.424127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1028/2004] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:50.427124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.431731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.437433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bbd63-86d8-4623-9fac-e7ae39a12eb5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.443730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.444403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.444691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.445108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.445741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.447768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.492098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 977/2005] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:50.493739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.525552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcbc69-c99a-4c75-9891-e38094e71616 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.598407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf0927-4ef6-4dd0-8820-5061ddf6f7ef admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.599523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.600203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.600480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.600752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.607397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.608664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.610014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.610694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] PATH_INFO: `/v3/projects/257c10eeb1ea455aa1a9d3bbddf39e26/users/26761c11bc7f479cbba8bcc4ea010252/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.635310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257c10eeb1ea455aa1a9d3bbddf39e26, user_id=26761c11bc7f479cbba8bcc4ea010252, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.641183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.667427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 978/2006] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:50 2022] PUT /identity/v3/projects/257c10eeb1ea455aa1a9d3bbddf39e26/users/26761c11bc7f479cbba8bcc4ea010252/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:50.668505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.683823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:50.699090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.701568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c44a32-5b95-4fca-b3d2-c938d97196a3 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.722880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1029/2007] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:50.725500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd32-e03e-4000-9106-0374215139ea None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.725829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd32-e03e-4000-9106-0374215139ea None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.726098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f875fd32-e03e-4000-9106-0374215139ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.771952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f875fd32-e03e-4000-9106-0374215139ea None None] MFA Rules not processed for user `26761c11bc7f479cbba8bcc4ea010252`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:50.773139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25cb0f-1841-4cba-a738-40e43e988f46 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.773869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f875fd32-e03e-4000-9106-0374215139ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.779557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.780647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.781178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.781542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.782696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.786445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.814220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 979/2008] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:50.815633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.828171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:50.855812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1030/2009] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:50.860185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875fd32-e03e-4000-9106-0374215139ea None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.876762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f875fd32-e03e-4000-9106-0374215139ea None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.885814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.888165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.888573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.889047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.890297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.893420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.901597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af18393-c451-444f-8b10-e3fc04c20a56 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.906142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.906920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.907190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.907467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.909834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:50.913508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1031/2010] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:50.915076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:50.930035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5c1c1-bbb3-44a8-bd3a-5e272acd5e2a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:50.935783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:50.936776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:50.937246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:50.937717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:50.938744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:50.941798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:50.985300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1032/2011] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:50 2022] POST /identity/v3/projects => generated 391 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:50.985300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.001220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.004130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.018016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 980/2012] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:51.019925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.042337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e39b9d-e254-4548-8398-1a96c5c299e6 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.065191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26189414-e7e6-42f9-80eb-fe2e7a698238 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.073819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.074503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.074808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.075094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.076375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.078042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.093631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1033/2013] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:51.099304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.116840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a0d87-cbae-4d9c-a21a-d1f4b6d0b307 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.126892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.129583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.129762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.130289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] PATH_INFO: `/v3/projects/2ee02dba06c5486ca1b28d2fd75a8636/users/e045d5a6d9674a52844c94d9d56c97dd/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.159000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ee02dba06c5486ca1b28d2fd75a8636, user_id=e045d5a6d9674a52844c94d9d56c97dd, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.159224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.159862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.160148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.160440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.161263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.163341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.164819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c501df-06b4-449a-bf84-93d15cf647c1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.189894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 981/2014] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:51 2022] POST /identity/v3/users => generated 302 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:51.193853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1034/2015] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:51 2022] PUT /identity/v3/projects/2ee02dba06c5486ca1b28d2fd75a8636/users/e045d5a6d9674a52844c94d9d56c97dd/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:51.204222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.208148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.208619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.208995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.219897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143b76-088e-473b-b8ad-ca0c2a398226 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.252154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] MFA Rules not processed for user `e045d5a6d9674a52844c94d9d56c97dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:51.281066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:51.301904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:51.302731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:51.324782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.325412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.325688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.325964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.327015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.328592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.354969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 982/2016] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:51.364003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.383275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4edb0fa-58b4-4858-97b7-7af5fdb9157c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.402647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.403908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.404400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.404880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] PATH_INFO: `/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99/users/d7de8a0eeb574b04b544d705bfaf649b/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.428797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b24a7af7f34f0ca3c3ac875c058a99, user_id=d7de8a0eeb574b04b544d705bfaf649b, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.432540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.462829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1035/2017] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:51.484449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 983/2018] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:51 2022] PUT /identity/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99/users/d7de8a0eeb574b04b544d705bfaf649b/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:51.492430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.492634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.494652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:51.514721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e39ee-b27c-4452-bb8f-115cae307b43 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.649268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.650297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.650830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.651285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.652098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.653994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.654319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a9b1b4-dd78-4419-a432-ce13f6f45cf8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.659561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.661673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.661962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.662237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.670776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1036/2019] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:51 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:51.676342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.699366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102daa2-fef9-4647-9599-035d5327d1c3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:51.707289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:51.708838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.709456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.709795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] PATH_INFO: `/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99/users/d7de8a0eeb574b04b544d705bfaf649b/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.712996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b24a7af7f34f0ca3c3ac875c058a99, user_id=d7de8a0eeb574b04b544d705bfaf649b, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.715973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb82e9f-db02-4a70-a93f-abad50029ef5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.733493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1037/2020] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:51 2022] PUT /identity/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99/users/d7de8a0eeb574b04b544d705bfaf649b/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:51.745593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:51.745848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:51.746132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:51.765016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:51.767348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:51.779038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] MFA Rules not processed for user `d7de8a0eeb574b04b544d705bfaf649b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:51.780301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:51.784323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 984/2021] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:00:51.880193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1038/2022] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:00:51.907471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:51.909354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:52.002819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57980172-c32e-4aab-87cd-8f0154b8ddd7 service glance] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:52.018961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:52.092183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:52.093188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:52.093655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:52.094166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:52.111217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:52.114284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:52.126170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 985/2023] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:51 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:52.193419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:52.206614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ee0b2b-1464-4f9c-ae51-d32f04aa5608 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:52.215614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:52.217767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:52.218176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:52.218583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:52.237147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:52.295591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:52.297263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fb013-21c9-4b45-9066-6f022ef0abc8 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:52.312506 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1039/2024] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:55.278337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:55.279654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:55.360191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c1d6c2-e76e-4e62-8d83-3a10d7bfa379 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:55.367505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:55.368412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:55.368787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:55.369372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] PATH_INFO: `/v3/users/1abd7907b483440883321b1249b49dcb` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:55.370818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abd7907b483440883321b1249b49dcb)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:55.373225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:55.448575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abd7907b483440883321b1249b49dcb'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:55.449320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abd7907b483440883321b1249b49dcb'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:55.465962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abd7907b483440883321b1249b49dcb'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:55.474145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abd7907b483440883321b1249b49dcb'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:55.481962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abd7907b483440883321b1249b49dcb'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:55.489616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 986/2025] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:55 2022] DELETE /identity/v3/users/1abd7907b483440883321b1249b49dcb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:55.495666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:55.496304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:55.496304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:55.525148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:55.526386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:55.589569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1040/2026] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:55.600789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:55.602340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:55.648317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:55.668417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d9e72-7345-4ff8-96ed-ce67ff6067ab admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:55.672128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:55.672740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:55.673017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:55.673284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:55.682911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:55.684384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:55.694829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 987/2027] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:56.111559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:56.114423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:56.138577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515ce7f-bea1-426d-ad5e-8bb525217a01 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:56.146782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:56.148146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:56.148700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:56.149563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] PATH_INFO: `/v3/projects/ba7a6d7160f24e589f1180696dccb60a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:56.151707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7a6d7160f24e589f1180696dccb60a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:56.154099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:56.231942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7a6d7160f24e589f1180696dccb60a', 'request_id': 'req-07c98013-50e2-4447-9cc0-de260561e665'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.242531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7a6d7160f24e589f1180696dccb60a', 'request_id': 'req-07c98013-50e2-4447-9cc0-de260561e665'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.249915 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7a6d7160f24e589f1180696dccb60a', 'request_id': 'req-07c98013-50e2-4447-9cc0-de260561e665'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.261792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] The token cache is being invalidate because project ba7a6d7160f24e589f1180696dccb60a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:56.262087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.262778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.264324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1041/2028] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:56 2022] DELETE /identity/v3/projects/ba7a6d7160f24e589f1180696dccb60a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:56.268480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:56.280022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ba42e-8eaf-40de-991d-5e64a13bb709 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:56.341797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:56.342430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:56.342723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:56.342991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] PATH_INFO: `/v3/users/bbccd5c09a294e11b3b1a32bc53c169d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:56.344291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbccd5c09a294e11b3b1a32bc53c169d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:56.346224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:56.400111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbccd5c09a294e11b3b1a32bc53c169d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.401052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbccd5c09a294e11b3b1a32bc53c169d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.415782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbccd5c09a294e11b3b1a32bc53c169d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.424788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbccd5c09a294e11b3b1a32bc53c169d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.433200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-47bfd300-5282-46c7-bb10-eeadcaaacfa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbccd5c09a294e11b3b1a32bc53c169d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.440994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 988/2029] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:56 2022] DELETE /identity/v3/users/bbccd5c09a294e11b3b1a32bc53c169d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:56.727671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:56.729322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:56.743341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c98013-50e2-4447-9cc0-de260561e665 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:56.793616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:56.820855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:56.821488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:56.821766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:56.822052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] PATH_INFO: `/v3/projects/1ba839a1f21f437c8f7125591f7ff8be` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:56.823172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ba839a1f21f437c8f7125591f7ff8be)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:56.825129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:56.870851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ba839a1f21f437c8f7125591f7ff8be', 'request_id': 'req-f1f75b71-cdd1-4298-b448-8804f83709ea'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.879560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ba839a1f21f437c8f7125591f7ff8be', 'request_id': 'req-f1f75b71-cdd1-4298-b448-8804f83709ea'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.888822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ba839a1f21f437c8f7125591f7ff8be', 'request_id': 'req-f1f75b71-cdd1-4298-b448-8804f83709ea'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.900065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] The token cache is being invalidate because project 1ba839a1f21f437c8f7125591f7ff8be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:56.900356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.901023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f1f75b71-cdd1-4298-b448-8804f83709ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:56.902517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1042/2030] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:56 2022] DELETE /identity/v3/projects/1ba839a1f21f437c8f7125591f7ff8be => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:56.907628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:56.907952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:56.908228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:56.928952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:56.930186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:56.984489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:57.008069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 989/2031] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:57.010953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.011312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.011648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.035019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:57.036952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:57.045111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1043/2032] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:57.049890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.061401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849e84f-8cf6-4be3-9c5d-856305ee1c46 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.066273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.066915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.067198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.067474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.068155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:00:57.068490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.070087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.082775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 990/2033] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:00:57.087288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.097296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74f809-2e17-4c7c-838c-a7d49ddc3c45 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.103011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.103530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.103829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.104102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.104733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.106631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.143767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1044/2034] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:57.148748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.159278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90848b-25e0-4c78-9b8f-697a8c8d59f0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.219397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.220082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.220367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.220643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.221305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.224042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.249728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 991/2035] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:57.255388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.270171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7360399d-1464-4fe3-b182-c1998eccd9b4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.281923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.281923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.281923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.281923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.282522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.282522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.292244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1045/2036] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:57.297037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.307573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f7da-7219-47b4-9b06-7f2264cf50ab admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.312895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.313654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.314020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.314419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.315140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.316822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c2082-fa9f-47fa-8872-0be4f15761a9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.328312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 992/2037] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:57.333244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.343847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdef607-0d8e-4966-9c74-ea5a5e7c0065 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.351809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.352967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.353443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.353911 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] PATH_INFO: `/v3/projects/bda9a10f85c74afbada7128f75051b1b/users/f48cca5aad9d4a3180c2647dd647dc46/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.371438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] RBAC: Authorizing `identity:create_grant(project_id=bda9a10f85c74afbada7128f75051b1b, user_id=f48cca5aad9d4a3180c2647dd647dc46, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.374435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.388240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1046/2038] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:57 2022] PUT /identity/v3/projects/bda9a10f85c74afbada7128f75051b1b/users/f48cca5aad9d4a3180c2647dd647dc46/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:57.400011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.400303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.400578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.431392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] MFA Rules not processed for user `f48cca5aad9d4a3180c2647dd647dc46`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:57.432754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:57.501825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 993/2039] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:57.506493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.569035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61a2964-2b96-4f93-8cf3-a6e729405bab admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.573769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.574375 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.574653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.574921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.575560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.577545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.587990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:57.609357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1047/2040] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:00:57.614504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.626376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec56e78e-ccf4-48d1-995e-b316568c9a2f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.679105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:57.691926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.692624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.692911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.693183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.693846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.695887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.717965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 994/2041] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:00:57.721899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.732415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c1719-3e84-4f84-8613-6bdbc01ea700 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.737329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.737951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.738236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.738631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.739276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.740917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.750987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1048/2042] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:00:57.756747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.770298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8ee3c-13ff-495d-9dcd-5baa78cea218 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.776458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.777179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.777513 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.777852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] PATH_INFO: `/v3/projects/093350d54516440ca42d153a598ffbfc/users/b5241d1d29dd4937b04e617d72cd4b0f/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.802864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=093350d54516440ca42d153a598ffbfc, user_id=b5241d1d29dd4937b04e617d72cd4b0f, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.805694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.820039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 995/2043] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:57 2022] PUT /identity/v3/projects/093350d54516440ca42d153a598ffbfc/users/b5241d1d29dd4937b04e617d72cd4b0f/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:57.824474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.836874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c07f-3bfe-4c80-b764-12513117524a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.897935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.898762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.899136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.899542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.900354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.902078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb3437e-fc9b-4478-8064-9b4d27e4a8db admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.912799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1049/2044] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:00:57.917648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:57.931779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90f0fe-24b4-491e-bd65-3677af7a42cb admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:57.936712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:57.937371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.937651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.937920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] PATH_INFO: `/v3/projects/093350d54516440ca42d153a598ffbfc/users/b5241d1d29dd4937b04e617d72cd4b0f/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.940060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=093350d54516440ca42d153a598ffbfc, user_id=b5241d1d29dd4937b04e617d72cd4b0f, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:57.943577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:57.956138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 996/2045] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:00:57 2022] PUT /identity/v3/projects/093350d54516440ca42d153a598ffbfc/users/b5241d1d29dd4937b04e617d72cd4b0f/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:57.967227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:57.967565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:57.967842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:57.998494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] MFA Rules not processed for user `b5241d1d29dd4937b04e617d72cd4b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:57.999728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:58.114955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1050/2046] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:58.126051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:58.128680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:58.203669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034c494-173b-4096-bcf6-83c1f3aa3e6f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:58.207594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:58.208423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:58.208779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:58.209142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:58.219858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:58.221464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dccfd1a-5948-451d-8333-632cb589ca67 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:58.232144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 997/2047] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:58.676937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:58.740224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c90cf9-bcfc-44d1-9784-8a7cb940c709 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:58.744850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:58.745487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:58.745772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:58.746048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] PATH_INFO: `/v3/users/f48cca5aad9d4a3180c2647dd647dc46` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:58.747190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48cca5aad9d4a3180c2647dd647dc46)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:58.749168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:58.750845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:58.801562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48cca5aad9d4a3180c2647dd647dc46'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:58.802288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48cca5aad9d4a3180c2647dd647dc46'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:58.810747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48cca5aad9d4a3180c2647dd647dc46'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:58.836264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48cca5aad9d4a3180c2647dd647dc46'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:58.839594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48cca5aad9d4a3180c2647dd647dc46'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:58.845778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1051/2048] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:58 2022] DELETE /identity/v3/users/f48cca5aad9d4a3180c2647dd647dc46 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:00:58.852643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:58.853149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:58.853254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:58.872888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:00:58.874298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:58.942957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 998/2049] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:00:58.954505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:58.956672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:59.016682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7544dd-d590-469a-a0cd-8a67d8b9c824 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:59.021512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:59.022133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:59.022416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:59.022700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:59.033630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:59.035127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:59.045897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1052/2050] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:00:59.372582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:59.375945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:00:59.398602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d65eaa-af4d-4e3c-ae42-9b409c6481d6 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:59.408452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:59.409457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:59.409800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:59.410153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] PATH_INFO: `/v3/projects/bda9a10f85c74afbada7128f75051b1b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:59.412946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bda9a10f85c74afbada7128f75051b1b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:59.416290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:59.481556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:59.492320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bda9a10f85c74afbada7128f75051b1b', 'request_id': 'req-ccda5665-57ae-4b2c-a6d4-70ef1c561801'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.503051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bda9a10f85c74afbada7128f75051b1b', 'request_id': 'req-ccda5665-57ae-4b2c-a6d4-70ef1c561801'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.511523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bda9a10f85c74afbada7128f75051b1b', 'request_id': 'req-ccda5665-57ae-4b2c-a6d4-70ef1c561801'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.530192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] The token cache is being invalidate because project bda9a10f85c74afbada7128f75051b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:00:59.530490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.531170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.532986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 999/2051] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:00:59 2022] DELETE /identity/v3/projects/bda9a10f85c74afbada7128f75051b1b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:00:59.538241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:00:59.553923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e797c8-30e3-41ee-8356-9444f0f8a8d8 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:00:59.620801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:00:59.621428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:00:59.621705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:00:59.621998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] PATH_INFO: `/v3/users/b5241d1d29dd4937b04e617d72cd4b0f` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:00:59.623108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5241d1d29dd4937b04e617d72cd4b0f)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:00:59.625074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:00:59.651743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:00:59.683209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5241d1d29dd4937b04e617d72cd4b0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.683905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5241d1d29dd4937b04e617d72cd4b0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.697097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5241d1d29dd4937b04e617d72cd4b0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.710797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5241d1d29dd4937b04e617d72cd4b0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.718085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-856448f7-9c82-44a5-86db-704a6849c023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5241d1d29dd4937b04e617d72cd4b0f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:00:59.726261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1053/2052] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:00:59 2022] DELETE /identity/v3/users/b5241d1d29dd4937b04e617d72cd4b0f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:00.032008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.033816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.050707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccda5665-57ae-4b2c-a6d4-70ef1c561801 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.122212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.122760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.123068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.123339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] PATH_INFO: `/v3/projects/093350d54516440ca42d153a598ffbfc` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.124514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093350d54516440ca42d153a598ffbfc)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.126418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.176799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093350d54516440ca42d153a598ffbfc', 'request_id': 'req-90f93ba8-0acd-45b5-9e93-afad55699a73'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:00.185432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093350d54516440ca42d153a598ffbfc', 'request_id': 'req-90f93ba8-0acd-45b5-9e93-afad55699a73'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:00.194771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093350d54516440ca42d153a598ffbfc', 'request_id': 'req-90f93ba8-0acd-45b5-9e93-afad55699a73'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:00.206013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] The token cache is being invalidate because project 093350d54516440ca42d153a598ffbfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:00.206295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:00.206972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-90f93ba8-0acd-45b5-9e93-afad55699a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:00.209227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1000/2053] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:00 2022] DELETE /identity/v3/projects/093350d54516440ca42d153a598ffbfc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:00.215399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.216183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.219289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.247560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:00.249692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.325212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1054/2054] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:00.332368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.332859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.333410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.360444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:00.361917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.372392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1001/2055] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:00.376795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.387572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37d4c9-c341-4fb0-b0cc-01fca6e70372 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.392317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.392927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.393206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.393481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.394131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:00.394440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.396113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.408396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1055/2056] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:00.413099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.422559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f81a-a3dc-4a36-b6aa-a79444c09563 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.428976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.429685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.430064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.430483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.431290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.433920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.473384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1002/2057] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:00.474735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.487758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2a36a-0684-4c8f-8386-7ed16988ee87 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.549710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.550330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.550605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.551611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.552260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.554208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.578149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1056/2058] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:00.583358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.594313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d2ec88-9915-4bde-8afa-36f27bcb45be admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.599286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.599930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.600344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.600489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.601126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.602729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.614523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1003/2059] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:00.619260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.630532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd7d6-10c5-4e57-834b-09f74ff09987 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.635428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.636158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.636496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.636843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.637533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.639273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9cff7-1aed-4bbd-89ee-f7b41a7549fd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.650634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1057/2060] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:00.655077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.665835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98467eec-c6d3-4f4d-ba0c-c69a213d585b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.670501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.671154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.671430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.671701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] PATH_INFO: `/v3/projects/c12e517411b94021b245fb89b6daed29/users/863ac1e3894449f68cf311f6a2995537/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.687061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12e517411b94021b245fb89b6daed29, user_id=863ac1e3894449f68cf311f6a2995537, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.689923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.699410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:00.705772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1004/2061] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:00 2022] PUT /identity/v3/projects/c12e517411b94021b245fb89b6daed29/users/863ac1e3894449f68cf311f6a2995537/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:00.711747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a3274-f358-4e61-8b04-3aeb7ddf4266 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.712185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a3274-f358-4e61-8b04-3aeb7ddf4266 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.712536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a3274-f358-4e61-8b04-3aeb7ddf4266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.726851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:00.733648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-150a3274-f358-4e61-8b04-3aeb7ddf4266 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:00.735705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-150a3274-f358-4e61-8b04-3aeb7ddf4266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.798982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1058/2062] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:00.810116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:00.811940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:00.874354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f886bfd-c3d9-414b-9b16-97131a24ac98 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:00.878040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:00.878694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:00.878926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:00.879228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:00.892552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:00.894001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:00.906258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1005/2063] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:01.842494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:01.843274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:01.843454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:01.877201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] MFA Rules not processed for user `863ac1e3894449f68cf311f6a2995537`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:01.878707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:01.963448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:01.966851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:02.003333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1059/2064] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:02.011259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.033993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f99b805-f8fe-4215-a102-6de0733e7930 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.050676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.052725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.053116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.053513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.056341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.058430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.060491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb11d76b-46b8-4d83-a29e-ea3a97bb6b3a service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.070135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.070135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.070135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.070135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.073114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:02.137576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1060/2065] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:02 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:02.141834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.169312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.172254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.191155 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1006/2066] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:02.246038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d605689-bbcf-4469-92ae-af485b248c8c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.252504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.253205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.253488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.253763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.254401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.256402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.282774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1061/2067] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:02 2022] POST /identity/v3/users => generated 293 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:02.284183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.300725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502ea43-9501-405f-83a7-086c354b29da service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.311055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.311865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.312293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.312702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.313594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.316168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.336439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1007/2068] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:02.340907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.353207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:02.357747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c653ae44-d770-494e-8c61-f3c9ab14e089 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.365938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.366766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.367101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.367695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] PATH_INFO: `/v3/projects/6bcab14060e94cd6b236026e9814749a/users/58a3ed135f254961ab72b5ac6cb1f059/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.387434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcab14060e94cd6b236026e9814749a, user_id=58a3ed135f254961ab72b5ac6cb1f059, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.390335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.408860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1062/2069] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:02 2022] PUT /identity/v3/projects/6bcab14060e94cd6b236026e9814749a/users/58a3ed135f254961ab72b5ac6cb1f059/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:02.413726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.424237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89795e6-3634-45d5-9482-278db7a7b494 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.480848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:02.506635 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.507259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.508592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.508883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.509541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.511150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68ebab3-0c38-4ef6-8235-3b79c0316225 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.528302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1008/2070] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:02.531511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:02.543496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdf0ed-bd4f-419f-aad5-86fb3a014027 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:02.554349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:02.555347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:02.555728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:02.556122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] PATH_INFO: `/v3/projects/6bcab14060e94cd6b236026e9814749a/users/58a3ed135f254961ab72b5ac6cb1f059/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:02.559513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bcab14060e94cd6b236026e9814749a, user_id=58a3ed135f254961ab72b5ac6cb1f059, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:02.563843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:02.577507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1063/2071] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:02 2022] PUT /identity/v3/projects/6bcab14060e94cd6b236026e9814749a/users/58a3ed135f254961ab72b5ac6cb1f059/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:03.579874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:03.580395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:03.580875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:03.633809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] MFA Rules not processed for user `58a3ed135f254961ab72b5ac6cb1f059`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:03.635245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:03.745478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1009/2072] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:03.758996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:03.849234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a10ea9-c17e-40b1-8d07-17219b0d96eb admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:03.856221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:03.859885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:03.859885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:03.859885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:03.929263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:03.930730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99957052-fc60-449b-b523-5395754fe13b service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:03.940024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1064/2073] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:13.745883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=45 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.827147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:13.828848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:13.914507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78832f40-249e-4939-aa86-b6226c269ec5 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:13.919347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:13.920029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:13.920334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:13.920614 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] PATH_INFO: `/v3/users/9000dd577a344dde85a882f2a2d51ea1` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:13.921729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9000dd577a344dde85a882f2a2d51ea1)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:13.923651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:13.935872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:13.984540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9000dd577a344dde85a882f2a2d51ea1'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:13.985544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9000dd577a344dde85a882f2a2d51ea1'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.004678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9000dd577a344dde85a882f2a2d51ea1'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.015165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9000dd577a344dde85a882f2a2d51ea1'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.030392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9000dd577a344dde85a882f2a2d51ea1'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.039853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1010/2074] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:13 2022] DELETE /identity/v3/users/9000dd577a344dde85a882f2a2d51ea1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:14.045987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.046361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.046665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.069725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:14.072125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:14.145054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1065/2075] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:14.156770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.158580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:14.218235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea99c009-6d25-4283-be66-fabc5e5ec78f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.221904 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.222540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.222798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.223069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.234041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.235827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.252179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1011/2076] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:14.569331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.570793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:14.590366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332629fe-0222-4679-8c98-e83a8c479082 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.595612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.596680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.597123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.597712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] PATH_INFO: `/v3/projects/59c9cf58e85a49b2a8db7c7c29367ada` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.600664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c9cf58e85a49b2a8db7c7c29367ada)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.602996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.655946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c9cf58e85a49b2a8db7c7c29367ada', 'request_id': 'req-3639ba4e-1580-4e99-9e4d-89335fbedd21'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.665549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c9cf58e85a49b2a8db7c7c29367ada', 'request_id': 'req-3639ba4e-1580-4e99-9e4d-89335fbedd21'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.672795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c9cf58e85a49b2a8db7c7c29367ada', 'request_id': 'req-3639ba4e-1580-4e99-9e4d-89335fbedd21'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.683530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] The token cache is being invalidate because project 59c9cf58e85a49b2a8db7c7c29367ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:14.683864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.684521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.686017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1066/2077] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:14 2022] DELETE /identity/v3/projects/59c9cf58e85a49b2a8db7c7c29367ada => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:14.690051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:14.700232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd878d-fb0c-47f0-bd24-98167411fd70 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:14.719716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:14.762160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:14.762825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:14.763095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:14.763368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] PATH_INFO: `/v3/users/f6c23810b76645079487f2757976851a` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:14.764567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c23810b76645079487f2757976851a)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:14.766551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:14.818565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c23810b76645079487f2757976851a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.819230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c23810b76645079487f2757976851a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.832047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c23810b76645079487f2757976851a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.840896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c23810b76645079487f2757976851a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.850080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c23810b76645079487f2757976851a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:14.856520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1012/2078] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:14 2022] DELETE /identity/v3/users/f6c23810b76645079487f2757976851a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:15.068564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.070529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:15.146758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639ba4e-1580-4e99-9e4d-89335fbedd21 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.167203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.170712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:15.198328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:15.213872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.214471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.214806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.215085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.217921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:15.269636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5c46b-3c34-4055-bd1c-6f1b3e3deb1c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.275537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.276226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.277234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.277458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] PATH_INFO: `/v3/projects/b01760e2d68240d9aeccfb440fd40d8b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.278660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01760e2d68240d9aeccfb440fd40d8b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.280821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.299050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.300646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f680a19-32cf-4bcf-8544-a3a86eb08ae4 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.315062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1067/2079] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:15.342642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:15.349757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01760e2d68240d9aeccfb440fd40d8b', 'request_id': 'req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:15.359992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01760e2d68240d9aeccfb440fd40d8b', 'request_id': 'req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:15.372176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01760e2d68240d9aeccfb440fd40d8b', 'request_id': 'req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:15.386404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] The token cache is being invalidate because project b01760e2d68240d9aeccfb440fd40d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:15.386816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:15.387618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-37eb29f1-b90e-4969-b1d0-0efc8a228ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:15.392668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1013/2080] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:15 2022] DELETE /identity/v3/projects/b01760e2d68240d9aeccfb440fd40d8b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:15.400026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.400590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.400947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.426109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:15.427954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:15.516399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1068/2081] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:15.524737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.525034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.525327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.550234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:15.551698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:15.560984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1014/2082] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:15.565589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.580388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27dcac-51cb-4c33-afe3-14f21a76450d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.595643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.596308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.596587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.597316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.597987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:15.598301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.600969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.616426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1069/2083] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:15.622310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.634334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234f4b4a-7fc2-482f-a207-84f132859ee1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.639168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.639767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.640068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.640370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.641043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.642947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.678485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1015/2084] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:15 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:15.683603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.701688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf508a-5b89-4a74-98a3-e9834ad6038b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.792415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.793204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.793565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.793937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.794716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.797555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.824241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1070/2085] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:15 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:15.828766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.849861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07199baa-4417-4d4e-8374-2a331081d275 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.854842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.855519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.855852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.856105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.856748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.858352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.871065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1016/2086] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:15.875783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.888145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2593b56-3b09-4737-ad35-08f97c1253f8 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.893307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.893916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.894202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.894475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.895125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.897648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0291a34-71e5-48b4-aca1-1ac72273cb6c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.899376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:15.918245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1071/2087] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:15.919491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:15.935289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c00212-846d-4cc2-8d39-963f075886b7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:15.940603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:15.941259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:15.941679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:15.941962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] PATH_INFO: `/v3/projects/ac770ae96a814f5cae8e60ba10ca9011/users/5b7ce2127ad2493db1876329194e0333/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:15.962643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac770ae96a814f5cae8e60ba10ca9011, user_id=5b7ce2127ad2493db1876329194e0333, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:15.965372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:15.980702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1017/2088] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:15 2022] PUT /identity/v3/projects/ac770ae96a814f5cae8e60ba10ca9011/users/5b7ce2127ad2493db1876329194e0333/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:16.000395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:16.000799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:16.001208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:16.035420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] MFA Rules not processed for user `5b7ce2127ad2493db1876329194e0333`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:16.036746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:16.108933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1072/2089] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:16.144699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:16.146627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:16.206473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5417275f-256e-41a8-aa2e-e878823822b8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:16.210395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:16.211115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:16.211310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:16.211578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:16.220883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:16.222333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d726d8-5a16-4063-bbd5-3dcd48dcaf5a service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:16.232080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1018/2090] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:16.952440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:16.954673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:17.026273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98a3cfd-7e3e-4d8c-9fc7-1f6d76c9287e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.031546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.032327 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.032614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.032889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] PATH_INFO: `/v3/users/26761c11bc7f479cbba8bcc4ea010252` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.034036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26761c11bc7f479cbba8bcc4ea010252)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.036013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.091761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26761c11bc7f479cbba8bcc4ea010252'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.092763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26761c11bc7f479cbba8bcc4ea010252'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.104637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26761c11bc7f479cbba8bcc4ea010252'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.122893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26761c11bc7f479cbba8bcc4ea010252'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.134053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26761c11bc7f479cbba8bcc4ea010252'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.142726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1073/2091] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:16 2022] DELETE /identity/v3/users/26761c11bc7f479cbba8bcc4ea010252 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:17.148963 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.149751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.150243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.183739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:17.185005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:17.289935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1019/2092] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:17.301075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.302740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:17.309713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.372683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06e9f0-a824-4c8a-b8d5-721706acfd6a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.376308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.376917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.377196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.377474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.387314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.388802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.399852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1074/2093] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:17.692614 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.695099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:17.716138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07626cff-d4fa-482c-9ebb-c98131660ca4 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.727334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.728613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.729029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.729454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] PATH_INFO: `/v3/projects/257c10eeb1ea455aa1a9d3bbddf39e26` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.731128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=257c10eeb1ea455aa1a9d3bbddf39e26)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.734288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.746971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:17.816832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257c10eeb1ea455aa1a9d3bbddf39e26', 'request_id': 'req-dad68ed2-2804-4f82-af45-c136d5697b9d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.827669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257c10eeb1ea455aa1a9d3bbddf39e26', 'request_id': 'req-dad68ed2-2804-4f82-af45-c136d5697b9d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.834749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257c10eeb1ea455aa1a9d3bbddf39e26', 'request_id': 'req-dad68ed2-2804-4f82-af45-c136d5697b9d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.847318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] The token cache is being invalidate because project 257c10eeb1ea455aa1a9d3bbddf39e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:17.847679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.848258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.849917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1020/2094] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:17 2022] DELETE /identity/v3/projects/257c10eeb1ea455aa1a9d3bbddf39e26 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:17.854200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:17.865067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9ad0c-4783-4750-88f4-c012399598ee service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:17.930206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:17.930928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:17.931209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:17.931508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] PATH_INFO: `/v3/users/d7de8a0eeb574b04b544d705bfaf649b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:17.932775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7de8a0eeb574b04b544d705bfaf649b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:17.934821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:17.991843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7de8a0eeb574b04b544d705bfaf649b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:17.992582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7de8a0eeb574b04b544d705bfaf649b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.001018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7de8a0eeb574b04b544d705bfaf649b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.015022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7de8a0eeb574b04b544d705bfaf649b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.022087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-87049448-a9ae-4451-bafb-f07c1ad7a1bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7de8a0eeb574b04b544d705bfaf649b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.028243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1075/2095] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:17 2022] DELETE /identity/v3/users/d7de8a0eeb574b04b544d705bfaf649b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:18.318642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.320509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.341532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad68ed2-2804-4f82-af45-c136d5697b9d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.408351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.408997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.409270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.409541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] PATH_INFO: `/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.410670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b24a7af7f34f0ca3c3ac875c058a99)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.413734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.470982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b24a7af7f34f0ca3c3ac875c058a99', 'request_id': 'req-cef7cf7a-0d78-4313-ae50-20cff37c24c4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.481127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b24a7af7f34f0ca3c3ac875c058a99', 'request_id': 'req-cef7cf7a-0d78-4313-ae50-20cff37c24c4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.490333 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b24a7af7f34f0ca3c3ac875c058a99', 'request_id': 'req-cef7cf7a-0d78-4313-ae50-20cff37c24c4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.501359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] The token cache is being invalidate because project 67b24a7af7f34f0ca3c3ac875c058a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:18.501673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.502320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-cef7cf7a-0d78-4313-ae50-20cff37c24c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:18.504197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1021/2096] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:18 2022] DELETE /identity/v3/projects/67b24a7af7f34f0ca3c3ac875c058a99 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:18.509412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.510226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.510381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.530653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:18.531903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.554014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.603603 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1076/2097] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:18.613850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.614208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.614718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.639165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:18.640810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.655611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1022/2098] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:18.657275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.673591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8112311-54d8-4e53-a768-59cec977ca8b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.681035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.682165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.682759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.683075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.684068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:18.684587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.686926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.713276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1077/2099] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:18.714668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.730161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b0e81-5c6b-4cce-ae24-578e0d0b7427 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.736516 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.737314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.737838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.738502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.739313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.741781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff6c6f-aadd-4bb4-8b04-f87cb2210c4d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.761142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:18.777428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1023/2100] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/projects => generated 383 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:18.782008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.792625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c29bc3-9fb7-4612-8f50-00b52fa3d335 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.897050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.897725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.898007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.898279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.898930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.901955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.946636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1078/2101] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:18.949591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722a931-c1ee-4ad5-8bce-af3f93bf417e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.949886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722a931-c1ee-4ad5-8bce-af3f93bf417e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.950361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722a931-c1ee-4ad5-8bce-af3f93bf417e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.952738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:18.963381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adbf528-f343-495b-8e49-d59a437669db admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:18.968358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:18.968969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:18.969255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:18.969529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:18.970187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:18.971819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:18.974949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f722a931-c1ee-4ad5-8bce-af3f93bf417e None None] MFA Rules not processed for user `5b7ce2127ad2493db1876329194e0333`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:18.977080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f722a931-c1ee-4ad5-8bce-af3f93bf417e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:18.984931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1079/2102] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:18.989144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.001134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e824903-1c38-4881-8f18-e9568a4d7ebb admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.007756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.008743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.009315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.009702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.010423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.012457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.029654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1080/2103] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:19.030738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.042051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be39432-1e86-4612-abae-dd80d0c771b3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.047542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.048376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.048771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.049161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] PATH_INFO: `/v3/projects/a1bc4b1ada9f46e69ebc7ab003e9ca49/users/93b9c2fd3c65450f838f949bff08572b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.065670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1bc4b1ada9f46e69ebc7ab003e9ca49, user_id=93b9c2fd3c65450f838f949bff08572b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.068879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.075646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1024/2104] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:19.084643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1081/2105] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:19 2022] PUT /identity/v3/projects/a1bc4b1ada9f46e69ebc7ab003e9ca49/users/93b9c2fd3c65450f838f949bff08572b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:19.095013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.095356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.095687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.135127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] MFA Rules not processed for user `93b9c2fd3c65450f838f949bff08572b`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:19.136825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:19.211383 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1025/2106] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:19.216518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.284168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:19.289637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566834d-c69c-420e-b1d9-9273d46b42fe admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.297049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.297746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.298202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.298572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.299324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.301434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.339253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1082/2107] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:19 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:19.343295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.355178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb27c5d-5105-422a-9b7f-416fb0647cd3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.422378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.422997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.423279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.423579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.424447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.426426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.453946 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1026/2108] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:19 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:19.458550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.470406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b59f4d-720a-47f5-a35a-b40bcfadf03d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.478040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.479065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.479628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.480227 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.482194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.484091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.499508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1083/2109] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:19.503717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.514355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c27dc9-6e8e-49c2-b2ed-4edd6c1b8448 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.519666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.520373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.520656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.521317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] PATH_INFO: `/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf/users/5c70ad0174d8419a8ea965ec5d14dd20/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.540461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00b8c0852734c0ea19cd74c2f548ddf, user_id=5c70ad0174d8419a8ea965ec5d14dd20, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.543664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.548412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:19.564071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1027/2110] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:19 2022] PUT /identity/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf/users/5c70ad0174d8419a8ea965ec5d14dd20/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:19.568970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.581568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e276a5-8cac-46e4-9fca-80b1336bbce5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.672894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.673664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.674050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.674425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.675387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.677837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b5cea-2e90-421b-975c-8592da75a859 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.689751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1084/2111] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:19 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:19.696319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.707853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54b977-86e6-47b4-932e-26d5cf6348ba admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.713013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.713688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.713992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.714446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] PATH_INFO: `/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf/users/5c70ad0174d8419a8ea965ec5d14dd20/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.717263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00b8c0852734c0ea19cd74c2f548ddf, user_id=5c70ad0174d8419a8ea965ec5d14dd20, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:19.720073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:19.736952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1028/2112] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:19 2022] PUT /identity/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf/users/5c70ad0174d8419a8ea965ec5d14dd20/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:19.744296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.744678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.745040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:19.778208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] MFA Rules not processed for user `5c70ad0174d8419a8ea965ec5d14dd20`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:19.779694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:19.861368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1085/2113] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:19.873570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:19.876154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:19.980011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda8446-ce44-4788-8c85-618dd0bf8fc1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:19.988854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:19.989521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:19.989820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:19.990232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:20.002037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:20.003663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218dff2-ad5d-4f92-9146-325e0f39dfa6 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:20.016251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1029/2114] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:20.645474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:20.698198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:20.711068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b28b6-f6f4-493b-89a8-acd348abf301 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:20.715719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:20.716412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:20.716694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:20.716973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] PATH_INFO: `/v3/users/93b9c2fd3c65450f838f949bff08572b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:20.718123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b9c2fd3c65450f838f949bff08572b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:20.720121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:20.778512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b9c2fd3c65450f838f949bff08572b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:20.779175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b9c2fd3c65450f838f949bff08572b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:20.790127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b9c2fd3c65450f838f949bff08572b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:20.805018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b9c2fd3c65450f838f949bff08572b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:20.812312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b9c2fd3c65450f838f949bff08572b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:20.818535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1086/2115] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:20 2022] DELETE /identity/v3/users/93b9c2fd3c65450f838f949bff08572b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:20.825267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:20.825960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:20.825960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:20.848165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:20.850477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:20.924605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1030/2116] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:20.934942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:20.936453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:20.996981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497db18-74c4-47e0-9c60-1a8785dfea5b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.000976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.001665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.002024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.002366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.016692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:21.018274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:21.032225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1087/2117] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:21.521521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:21.523401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:21.541645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4238bf-f3b4-4307-9031-c11b191afe0b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.547468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.548170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.548456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.548731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] PATH_INFO: `/v3/projects/a1bc4b1ada9f46e69ebc7ab003e9ca49` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.550004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1bc4b1ada9f46e69ebc7ab003e9ca49)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:21.551928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:21.605522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1bc4b1ada9f46e69ebc7ab003e9ca49', 'request_id': 'req-f3750b95-2c0c-47ae-842c-5345c586f16d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.615141 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1bc4b1ada9f46e69ebc7ab003e9ca49', 'request_id': 'req-f3750b95-2c0c-47ae-842c-5345c586f16d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.622071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1bc4b1ada9f46e69ebc7ab003e9ca49', 'request_id': 'req-f3750b95-2c0c-47ae-842c-5345c586f16d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.633431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] The token cache is being invalidate because project a1bc4b1ada9f46e69ebc7ab003e9ca49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:21.633721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.634387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.635969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1031/2118] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:21 2022] DELETE /identity/v3/projects/a1bc4b1ada9f46e69ebc7ab003e9ca49 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:21.640038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:21.650752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f775-f723-46af-94c6-f5fded1e54e9 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:21.711417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:21.711996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:21.712277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:21.712550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] PATH_INFO: `/v3/users/5c70ad0174d8419a8ea965ec5d14dd20` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:21.713677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c70ad0174d8419a8ea965ec5d14dd20)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:21.715616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:21.731504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:21.764647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c70ad0174d8419a8ea965ec5d14dd20'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.765301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c70ad0174d8419a8ea965ec5d14dd20'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.773341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c70ad0174d8419a8ea965ec5d14dd20'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.785979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c70ad0174d8419a8ea965ec5d14dd20'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.794085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8c96f01b-a97d-4883-9191-cb1e59339ba5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c70ad0174d8419a8ea965ec5d14dd20'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:21.800265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1088/2119] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:21 2022] DELETE /identity/v3/users/5c70ad0174d8419a8ea965ec5d14dd20 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:22.116845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.119121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:22.134813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3750b95-2c0c-47ae-842c-5345c586f16d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.140765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.200666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.201310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.201589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.201865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] PATH_INFO: `/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.203021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00b8c0852734c0ea19cd74c2f548ddf)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.205040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.255315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00b8c0852734c0ea19cd74c2f548ddf', 'request_id': 'req-e23efa6f-4208-43f7-b98b-c3a11f509287'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:22.264703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00b8c0852734c0ea19cd74c2f548ddf', 'request_id': 'req-e23efa6f-4208-43f7-b98b-c3a11f509287'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:22.272305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00b8c0852734c0ea19cd74c2f548ddf', 'request_id': 'req-e23efa6f-4208-43f7-b98b-c3a11f509287'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:22.283336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] The token cache is being invalidate because project b00b8c0852734c0ea19cd74c2f548ddf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:22.283623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:22.284311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e23efa6f-4208-43f7-b98b-c3a11f509287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:22.285791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1032/2120] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:22 2022] DELETE /identity/v3/projects/b00b8c0852734c0ea19cd74c2f548ddf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:22.291364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.291700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.291997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.312963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:22.314213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:22.377540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1089/2121] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:22.386917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.387215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.387493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.410780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:22.413204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:22.421745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1033/2122] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:22.426307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.440985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d9feb-aa34-4636-93ed-ffe10e7707c2 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.446578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.447380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.447739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.448148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.448897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:22.449284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.451470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.464459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1090/2123] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:22.468918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.478891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b988582-f279-4470-ac5c-b04e6b95c9b5 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.483605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.484233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.484511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.484781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.485413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.487308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.516523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1034/2124] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:22.520935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.532352 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f824a5-e0f6-4a9e-a39c-c350c423b2fa admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.594008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.594348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.594616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.594840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.595472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.597852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.626447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1091/2125] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:22.627868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.640038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b297ae3-eb66-4dd2-9252-c5cd069c472e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.645335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.645948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.647047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.647367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.648255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.649855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.661219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1035/2126] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:22.665296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.675356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e93-9a70-4115-8091-a43170f20767 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.680704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.681306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.681583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.681853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.682494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.684148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07777e5-0d95-4f30-b993-36352fbee5b8 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.689517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.696954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1092/2127] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:22.700468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.711191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff062e2-d2b8-4b28-87db-8e474020ef41 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.718313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.719422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.719966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.720463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] PATH_INFO: `/v3/projects/409eee2dfd584b9e91a8ea57cd85b0ae/users/5131a439321346119af671cff4cab1ee/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.746490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=409eee2dfd584b9e91a8ea57cd85b0ae, user_id=5131a439321346119af671cff4cab1ee, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.751812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:22.772460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1036/2128] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:22 2022] PUT /identity/v3/projects/409eee2dfd584b9e91a8ea57cd85b0ae/users/5131a439321346119af671cff4cab1ee/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:22.782762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.783049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.783322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.814582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] MFA Rules not processed for user `5131a439321346119af671cff4cab1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:22.816244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:22.888060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1093/2129] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:22.892567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:22.941040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:22.968057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdce928-ed51-4c9a-88be-a58e3a9072b6 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:22.973042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:22.973792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:22.975011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:22.975365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:22.976212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:22.978337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.013023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1037/2130] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:22 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:23.014274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.026452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad0f4b-6eec-4545-9c60-00210ab1f5f4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.087988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.088629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.088934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.089194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.089825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.091834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.120586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1094/2131] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:23 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:23.122527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.147017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442cbc4-529a-4039-8a5c-327632901340 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.166404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.167033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.167309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.167578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.168265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.169921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.182458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1038/2132] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:23.186856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.198484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846670-fdc3-456b-b579-3c42df81c1ac admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.204202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.204870 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.205149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.205427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] PATH_INFO: `/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3/users/4617d89ba30945e68cf98f8dda1af793/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.222909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8dbf88a10047a9b5b8719388dcbac3, user_id=4617d89ba30945e68cf98f8dda1af793, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.225819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.243628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1095/2133] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:23 2022] PUT /identity/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3/users/4617d89ba30945e68cf98f8dda1af793/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:23.247737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.258467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff1325-4e0f-465a-a33e-07f420eca9cb admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.321576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.322220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.322550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.322773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.323419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.325068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a338091-5baf-4ab8-93db-b9d0ff17f53d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.337607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1039/2134] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:23.341961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.353826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4616e5-e167-4cbd-85f3-7877b81f96d2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.358559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.359222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.359508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.359780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] PATH_INFO: `/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3/users/4617d89ba30945e68cf98f8dda1af793/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.361722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8dbf88a10047a9b5b8719388dcbac3, user_id=4617d89ba30945e68cf98f8dda1af793, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.364712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.377464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1096/2135] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:23 2022] PUT /identity/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3/users/4617d89ba30945e68cf98f8dda1af793/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:23.389685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.389981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.390272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.422782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] MFA Rules not processed for user `4617d89ba30945e68cf98f8dda1af793`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:23.424057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:23.510595 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1040/2136] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:23.523050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.524794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:23.576012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:23.589710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682d89da-5b38-422d-99bc-3609bad33336 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.593560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.594167 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.594442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.594713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.604562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.606056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.621654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1097/2137] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:23.713764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:23.723397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5889c2c-8323-44d0-9c7d-60fda4c391e0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:23.727154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:23.727760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:23.728078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:23.728352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:23.760171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:23.789093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:23.791397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbc5a-1290-4e0f-bf90-04559f01546d service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:23.802268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1041/2138] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:24.085175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:24.151799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4b2a2-dcc6-43a3-b9c6-cfeda7412ec8 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:24.156461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:24.161674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.161674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.161674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] PATH_INFO: `/v3/users/5131a439321346119af671cff4cab1ee` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.161674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5131a439321346119af671cff4cab1ee)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:24.161674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:24.230367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5131a439321346119af671cff4cab1ee'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.231101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5131a439321346119af671cff4cab1ee'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.242423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5131a439321346119af671cff4cab1ee'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.268722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5131a439321346119af671cff4cab1ee'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.281824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5131a439321346119af671cff4cab1ee'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.290398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1098/2139] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:24 2022] DELETE /identity/v3/users/5131a439321346119af671cff4cab1ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:24.296318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.296982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.296982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.338627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:24.340476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:24.409199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1042/2140] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:24.419285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:24.420767 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:24.480621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fddbe7-753b-4125-ac81-f2c0b8ae744f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:24.484280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:24.484890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.485169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.485439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.495094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:24.496559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:24.506542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1099/2141] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:24.804319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:24.805859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:24.822323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157c89c-9e7e-4d9f-a23e-2cb83177fd13 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:24.827292 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:24.828165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:24.828498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:24.828726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] PATH_INFO: `/v3/projects/409eee2dfd584b9e91a8ea57cd85b0ae` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:24.829877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=409eee2dfd584b9e91a8ea57cd85b0ae)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:24.832893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:24.888389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '409eee2dfd584b9e91a8ea57cd85b0ae', 'request_id': 'req-d952f832-39f7-4e83-86b6-3a9674a958a8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.898144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '409eee2dfd584b9e91a8ea57cd85b0ae', 'request_id': 'req-d952f832-39f7-4e83-86b6-3a9674a958a8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.905546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '409eee2dfd584b9e91a8ea57cd85b0ae', 'request_id': 'req-d952f832-39f7-4e83-86b6-3a9674a958a8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.917342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] The token cache is being invalidate because project 409eee2dfd584b9e91a8ea57cd85b0ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:24.918045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.920274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:24.922452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1043/2142] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:24 2022] DELETE /identity/v3/projects/409eee2dfd584b9e91a8ea57cd85b0ae => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:24.927672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:24.938185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1b06-128a-4b74-bcaf-96ab4c599fd2 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:24.942224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:25.001371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:25.002007 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.002290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.002568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] PATH_INFO: `/v3/users/4617d89ba30945e68cf98f8dda1af793` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.003707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4617d89ba30945e68cf98f8dda1af793)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:25.006366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:25.054724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4617d89ba30945e68cf98f8dda1af793'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.055404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4617d89ba30945e68cf98f8dda1af793'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.064668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4617d89ba30945e68cf98f8dda1af793'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.078713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4617d89ba30945e68cf98f8dda1af793'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.087443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-31b367b3-a901-46a6-8e33-b66f87f3c71a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4617d89ba30945e68cf98f8dda1af793'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.093842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1100/2143] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:24 2022] DELETE /identity/v3/users/4617d89ba30945e68cf98f8dda1af793 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:25.412923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.414518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:25.430741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d952f832-39f7-4e83-86b6-3a9674a958a8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:25.488258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:25.499976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:25.500871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.501158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.501462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] PATH_INFO: `/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.502645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8dbf88a10047a9b5b8719388dcbac3)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:25.504702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:25.556989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8dbf88a10047a9b5b8719388dcbac3', 'request_id': 'req-ec5199dc-9c04-45ab-b266-b500e24a9fc4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.567354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8dbf88a10047a9b5b8719388dcbac3', 'request_id': 'req-ec5199dc-9c04-45ab-b266-b500e24a9fc4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.579465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8dbf88a10047a9b5b8719388dcbac3', 'request_id': 'req-ec5199dc-9c04-45ab-b266-b500e24a9fc4'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.595577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] The token cache is being invalidate because project 2a8dbf88a10047a9b5b8719388dcbac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:25.595851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.596495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ec5199dc-9c04-45ab-b266-b500e24a9fc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:25.599901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1044/2144] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:25 2022] DELETE /identity/v3/projects/2a8dbf88a10047a9b5b8719388dcbac3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:25.606798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.607161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.607505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.652582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:25.653885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:25.743632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1101/2145] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:25.751721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.752031 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.752334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.776664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:25.778148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:25.786712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1045/2146] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:25.792107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.802145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3aea45-2cce-4f38-ae1d-ec5f317d1e0a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:25.807712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:25.808508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.808798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.809086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.809744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:25.810049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:25.811674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:25.830131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1102/2147] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:25.831319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.843246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f71e51-b4ef-4c59-acfc-fabe1c662dac None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:25.848107 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:25.848665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:25.848941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:25.849210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:25.850608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:25.852556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:25.880612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1046/2148] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:25 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:25.885102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:25.897151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3118-22a7-4995-b06f-85b5acd101c6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:25.907487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.027555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.034865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.034865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.034865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.034865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.038660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.070676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1103/2149] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:25 2022] POST /identity/v3/users => generated 281 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:26.075172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.087629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefccdc9-aad4-487a-b885-79c4a8846718 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.093011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.093737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.094029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.094302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.095367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.097248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.110775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1047/2150] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:26.122495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.151747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aedd22-fc64-4903-adcd-45e7408b0437 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.167355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.168122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.168457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.168820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.170008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.173254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aafb39-3648-4a92-81bf-68886d80dfb9 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.195116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1104/2151] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:26.198804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.212939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce118dab-1cd0-4981-9d9d-1f1612d62971 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.220540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.221221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.221497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.221773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] PATH_INFO: `/v3/projects/d734b6a710e64e2db20f052c56247d12/users/eae709e5b2534fed8ba6907134b47e9a/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.241906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d734b6a710e64e2db20f052c56247d12, user_id=eae709e5b2534fed8ba6907134b47e9a, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.244981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.261654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1048/2152] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:26 2022] PUT /identity/v3/projects/d734b6a710e64e2db20f052c56247d12/users/eae709e5b2534fed8ba6907134b47e9a/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:26.273694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62285627-a8fb-40fe-8868-b854312add2c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.274147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62285627-a8fb-40fe-8868-b854312add2c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.274598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62285627-a8fb-40fe-8868-b854312add2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.309061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-62285627-a8fb-40fe-8868-b854312add2c None None] MFA Rules not processed for user `eae709e5b2534fed8ba6907134b47e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:26.310650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-62285627-a8fb-40fe-8868-b854312add2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:26.388437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1105/2153] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:26.424494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.428005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:26.530909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e6a2-d63e-438e-ae73-c4c4934a979c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.534642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.535260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.535542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.535847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.545685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.547190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c21955c-e834-4fd5-95ba-43f5d1a93d94 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.562046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1049/2154] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:26.581442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62285627-a8fb-40fe-8868-b854312add2c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.648566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62285627-a8fb-40fe-8868-b854312add2c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.655192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.655956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.656127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.656404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] PATH_INFO: `/v3/users/eae709e5b2534fed8ba6907134b47e9a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.657579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae709e5b2534fed8ba6907134b47e9a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.659534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.692858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.728850 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae709e5b2534fed8ba6907134b47e9a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:26.730621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae709e5b2534fed8ba6907134b47e9a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:26.745373 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae709e5b2534fed8ba6907134b47e9a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:26.767630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae709e5b2534fed8ba6907134b47e9a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:26.783835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae709e5b2534fed8ba6907134b47e9a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:26.791147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1106/2155] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:26 2022] DELETE /identity/v3/users/eae709e5b2534fed8ba6907134b47e9a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:26.797129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.797704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.798052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.821352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:26.822600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:26.841409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:26.894867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1050/2156] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:26.905993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:26.907571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:26.968833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a50624-5d14-4ee8-b58f-84e4f57f5c02 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:26.972535 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:26.973137 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:26.973412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:26.973679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:26.984044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:26.985477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8b556-984b-4efe-bbbb-cc95bd05fe00 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:26.996526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1107/2157] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:27.397046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.399558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:27.421705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e02729-0360-4484-9419-7004f41e7893 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.436664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.437977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.438265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.438541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] PATH_INFO: `/v3/projects/d734b6a710e64e2db20f052c56247d12` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.439739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d734b6a710e64e2db20f052c56247d12)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.441934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.504809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd734b6a710e64e2db20f052c56247d12', 'request_id': 'req-19b5d006-3415-4ad3-86f9-bd4a89714c44'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:27.512465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd734b6a710e64e2db20f052c56247d12', 'request_id': 'req-19b5d006-3415-4ad3-86f9-bd4a89714c44'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:27.521895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd734b6a710e64e2db20f052c56247d12', 'request_id': 'req-19b5d006-3415-4ad3-86f9-bd4a89714c44'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:27.536845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] The token cache is being invalidate because project d734b6a710e64e2db20f052c56247d12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:27.538252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:27.539131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-19b5d006-3415-4ad3-86f9-bd4a89714c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:27.541566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1051/2158] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:27 2022] DELETE /identity/v3/projects/d734b6a710e64e2db20f052c56247d12 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:27.546470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.546949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.547197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.567940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:27.569356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:27.637656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1108/2159] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:27.647129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.647408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.647652 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.668519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:27.669656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:27.677734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1052/2160] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:27.682253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.692978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15e27f7-ff03-469a-a122-dad2f86a0d63 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.698934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.699562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.699888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.700441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.701612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:27.701964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.703572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.718128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1109/2161] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:27.722331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.731971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f65dc-49e8-4446-8b3e-0c971180d1c2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.737070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.737693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.737969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.738238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.739147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.741115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.774347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1053/2162] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:27 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:27.778523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.788814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d790ed3-726d-4bab-b782-50f923963c32 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.824828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:27.865880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.866705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.866895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.867174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.867856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.869999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.897743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1110/2163] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:27 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:27.901772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.913572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffbcfa-f2a9-4c89-a96d-b8a0331c4003 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.918672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.919302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.919762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.920172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.921051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.922770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.936954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1054/2164] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:27.938385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.949105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d3d3f-454d-489d-a096-eb9ab2335de4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.953844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.954449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.954735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.955009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:27.955653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:27.957308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b115fec5-f453-4ef7-816b-6263f17d8e3e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:27.969063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1111/2165] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:27.973268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:27.983707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bf51d-1639-47ce-9f96-bdb0d4e04b09 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:27.988581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:27.989590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:27.989786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:27.990565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] PATH_INFO: `/v3/projects/9b6413b980ac4937b6ddc13988736b24/users/c743ef94052447bb8535cc13b328cfc6/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:28.006797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6413b980ac4937b6ddc13988736b24, user_id=c743ef94052447bb8535cc13b328cfc6, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:28.009957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:28.031104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1055/2166] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:27 2022] PUT /identity/v3/projects/9b6413b980ac4937b6ddc13988736b24/users/c743ef94052447bb8535cc13b328cfc6/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:28.042550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:28.042842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:28.043120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:28.074863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] MFA Rules not processed for user `c743ef94052447bb8535cc13b328cfc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:28.076316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:28.153490 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1112/2167] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:28.165060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:28.166890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:28.214840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:28.228277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01feaac6-a8bc-4ce4-98a6-a7f9785fe5c0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:28.231967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:28.232572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:28.232852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:28.233128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:28.243187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:28.244724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c7cc2-fc1e-485b-89ba-276d92ad0fe3 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:28.254059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1056/2168] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:38.150308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:38.249959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:43.796985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:43.798747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:43.906297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cbf6c-321e-4547-948e-3e6481f434bc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:43.912032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:43.913454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:43.913738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:43.914013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] PATH_INFO: `/v3/users/7e9e0768a9894388a01d37bb2aa340aa` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:43.922852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9e0768a9894388a01d37bb2aa340aa)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:43.924888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:43.980003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9e0768a9894388a01d37bb2aa340aa'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:43.980687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9e0768a9894388a01d37bb2aa340aa'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:43.989293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9e0768a9894388a01d37bb2aa340aa'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.003652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9e0768a9894388a01d37bb2aa340aa'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.012861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9e0768a9894388a01d37bb2aa340aa'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.019713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1113/2169] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:43 2022] DELETE /identity/v3/users/7e9e0768a9894388a01d37bb2aa340aa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:44.024736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:44.025199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:44.025299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:44.045701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:44.047144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:44.115051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1057/2170] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:44.125663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:44.127189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:44.201075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b5bdf-fb94-48a6-b19f-ccb2d045061b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:44.204726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:44.205326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:44.205609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:44.205880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:44.214702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:44.216165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:44.225326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1114/2171] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:44.518440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:44.520008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:44.532835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbedf6d0-8eb0-48b6-9470-b2953e625032 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:44.537651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:44.538284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:44.538567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:44.538848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] PATH_INFO: `/v3/projects/36ae13d7131b4bdfbef1cb92a750dda9` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:44.540575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36ae13d7131b4bdfbef1cb92a750dda9)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:44.542703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:44.589989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ae13d7131b4bdfbef1cb92a750dda9', 'request_id': 'req-46196272-db45-40f8-bb0e-f8fd3a927b48'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.597564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ae13d7131b4bdfbef1cb92a750dda9', 'request_id': 'req-46196272-db45-40f8-bb0e-f8fd3a927b48'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.604029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ae13d7131b4bdfbef1cb92a750dda9', 'request_id': 'req-46196272-db45-40f8-bb0e-f8fd3a927b48'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.614160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] The token cache is being invalidate because project 36ae13d7131b4bdfbef1cb92a750dda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:44.614448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.615347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.616849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1058/2172] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:44 2022] DELETE /identity/v3/projects/36ae13d7131b4bdfbef1cb92a750dda9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:44.621486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:44.631087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6072-d051-4e17-bc77-6087d0e272ef service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:44.650963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:44.692116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:44.692804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:44.693029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:44.693304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] PATH_INFO: `/v3/users/82395f2e0c5c417ea72f722e290c8bac` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:44.694443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82395f2e0c5c417ea72f722e290c8bac)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:44.697180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:44.742540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82395f2e0c5c417ea72f722e290c8bac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.743970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82395f2e0c5c417ea72f722e290c8bac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.751351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82395f2e0c5c417ea72f722e290c8bac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.764385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82395f2e0c5c417ea72f722e290c8bac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.771161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ff2191bf-2e0c-4163-a44d-3b69cd387352 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82395f2e0c5c417ea72f722e290c8bac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:44.777210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1115/2173] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:44 2022] DELETE /identity/v3/users/82395f2e0c5c417ea72f722e290c8bac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:45.053965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.055720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:45.069163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46196272-db45-40f8-bb0e-f8fd3a927b48 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.129014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.129600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.129878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.130159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] PATH_INFO: `/v3/projects/abba5d090bea476d93d0f554ad37b524` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.131326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] RBAC: Authorizing `identity:delete_project(project_id=abba5d090bea476d93d0f554ad37b524)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.133871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.165094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:45.182692 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abba5d090bea476d93d0f554ad37b524', 'request_id': 'req-ecee1c23-0135-4cd7-9467-2b1cdb26deed'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:45.191284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abba5d090bea476d93d0f554ad37b524', 'request_id': 'req-ecee1c23-0135-4cd7-9467-2b1cdb26deed'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:45.203022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abba5d090bea476d93d0f554ad37b524', 'request_id': 'req-ecee1c23-0135-4cd7-9467-2b1cdb26deed'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:45.219169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] The token cache is being invalidate because project abba5d090bea476d93d0f554ad37b524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:45.219456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:45.220135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ecee1c23-0135-4cd7-9467-2b1cdb26deed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:45.224933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1059/2174] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:45 2022] DELETE /identity/v3/projects/abba5d090bea476d93d0f554ad37b524 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:45.226500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.226961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.227699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.248334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:45.249572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:45.316371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1116/2175] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:45.325589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.325913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.326840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.347608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:45.349001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:45.359139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1060/2176] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:45.363307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.374791 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762b14-ce37-446e-a2f9-e2a9cdcce8dc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.379362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.379985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.380266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.380533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.381176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:45.381479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.383065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.395124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1117/2177] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:45.399996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.409784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aba07f-deb1-4240-a619-3786f384cae0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.414555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.415674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.415985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.416261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.416893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.418818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.451632 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1061/2178] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:45.455887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.466692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eee0f70-2320-4c63-829d-ba442d097854 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.492108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:45.525770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.526368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.526649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.526921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.527555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.529665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.551195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1118/2179] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:45.556171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.566617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552536ca-b2dd-4382-a245-ca9a679eb6a8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.572209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.572805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.573079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.573396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.573977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.575572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.585801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1062/2180] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:45.590494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.600716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa09ae2-e038-4013-835f-99268bf7580e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.605765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.606358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.606634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.606900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.607533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.609158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc570e0-5d69-4f4b-b28a-5c4dbf58ef3c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.620467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1119/2181] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:45.624158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.635422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bae0de-cd14-4bad-88b1-bdb24f8ca9db admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.640024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.640670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.640943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.641211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] PATH_INFO: `/v3/projects/11ea1e90109743dfb0115814a6726c55/users/9e11cd6fed78422c887f79d8b2888738/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.656562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ea1e90109743dfb0115814a6726c55, user_id=9e11cd6fed78422c887f79d8b2888738, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.659382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.673291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1063/2182] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:45 2022] PUT /identity/v3/projects/11ea1e90109743dfb0115814a6726c55/users/9e11cd6fed78422c887f79d8b2888738/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:45.684084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.684380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.684644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.712068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] MFA Rules not processed for user `9e11cd6fed78422c887f79d8b2888738`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:45.713339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:45.778579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1120/2183] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:45.783247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.846686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc9b2b-d548-408f-9533-50d29bf07041 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.851336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.852607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.852915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.853208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.853846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.855764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.883116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1064/2184] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:45.887298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:45.899368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d525dbe2-e2d2-43be-b7ea-bafb6dff6f66 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:45.955078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:45.955687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:45.955997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:45.956276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:45.956894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:45.958800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:45.982747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1121/2185] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:45 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:45.986974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.000825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64cf98-6652-4703-a7d6-2131bcb729e3 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.017248 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.017899 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.018179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.018451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.019086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.020727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.032328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1065/2186] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:46.036556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.051232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820c985-e736-4519-ac0d-2e2800c0446f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.056998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.057652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.057929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.058201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] PATH_INFO: `/v3/projects/365d9d95597f450d98547a9c4083c2b4/users/a4e4a066d2534edebc8c181b2fa22bb3/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.073790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365d9d95597f450d98547a9c4083c2b4, user_id=a4e4a066d2534edebc8c181b2fa22bb3, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.076483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.090471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1122/2187] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:46 2022] PUT /identity/v3/projects/365d9d95597f450d98547a9c4083c2b4/users/a4e4a066d2534edebc8c181b2fa22bb3/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:46.094370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.105636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6166dee9-f4a2-4a24-9055-5f3368dea816 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.125962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:46.167739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.168497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.168782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.169059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.169775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.171431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe17bc-641c-4eae-8227-162c66b3715f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.182004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1066/2188] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:46.186191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.191679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:46.198831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc5417-420e-433e-ac65-148d0a7a1dc2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.203728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.204413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.204699 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.204978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] PATH_INFO: `/v3/projects/365d9d95597f450d98547a9c4083c2b4/users/a4e4a066d2534edebc8c181b2fa22bb3/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.207168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365d9d95597f450d98547a9c4083c2b4, user_id=a4e4a066d2534edebc8c181b2fa22bb3, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.209879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.220667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1123/2189] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:46 2022] PUT /identity/v3/projects/365d9d95597f450d98547a9c4083c2b4/users/a4e4a066d2534edebc8c181b2fa22bb3/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:46.231987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.232210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.232496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.260882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] MFA Rules not processed for user `a4e4a066d2534edebc8c181b2fa22bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:46.262101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:46.330938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1067/2190] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:46.343341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.345134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:46.405455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0277cd1-48f7-40fb-b04a-1253c4f475a4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.409146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.409755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.410033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.410302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.421181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.422608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df27cb-ed6d-4890-bfff-f37f348dd206 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.433025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1124/2191] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:46.465910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.550854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db9280-85bc-4b05-bd03-b9cb448b2c41 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.556149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.556897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.557281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.557663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] PATH_INFO: `/v3/users/9e11cd6fed78422c887f79d8b2888738` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.559252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e11cd6fed78422c887f79d8b2888738)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.561238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.611821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e11cd6fed78422c887f79d8b2888738'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:46.612484 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e11cd6fed78422c887f79d8b2888738'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:46.613604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:46.626379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e11cd6fed78422c887f79d8b2888738'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:46.634015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e11cd6fed78422c887f79d8b2888738'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:46.640868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e11cd6fed78422c887f79d8b2888738'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:46.646716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1068/2192] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:46 2022] DELETE /identity/v3/users/9e11cd6fed78422c887f79d8b2888738 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:46.651930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.652222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.652502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.671457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:46.672742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:46.737257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1125/2193] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:46.749180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:46.750630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:46.808413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9652f6a7-33d6-4a6f-8fbe-28d655da5f4d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:46.812169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:46.812779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:46.813089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:46.813342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:46.823322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:46.824833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:46.834492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1069/2194] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:47.122052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:47.123576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:47.136406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f88884-e7ae-42a6-8265-5a845a18599e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:47.141101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:47.141773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:47.142056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:47.142334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] PATH_INFO: `/v3/projects/11ea1e90109743dfb0115814a6726c55` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:47.143470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ea1e90109743dfb0115814a6726c55)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:47.145433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:47.199982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ea1e90109743dfb0115814a6726c55', 'request_id': 'req-d48a87b8-6b32-426a-8ffb-3831ddf07c63'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.208364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ea1e90109743dfb0115814a6726c55', 'request_id': 'req-d48a87b8-6b32-426a-8ffb-3831ddf07c63'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.216293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ea1e90109743dfb0115814a6726c55', 'request_id': 'req-d48a87b8-6b32-426a-8ffb-3831ddf07c63'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.227353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] The token cache is being invalidate because project 11ea1e90109743dfb0115814a6726c55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:47.227642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.228342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.229922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1126/2195] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:47 2022] DELETE /identity/v3/projects/11ea1e90109743dfb0115814a6726c55 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:47.234042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:47.245127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7343b8eb-a424-4577-b514-bee596c9783f service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:47.304587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:47.305305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:47.305581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:47.305852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] PATH_INFO: `/v3/users/a4e4a066d2534edebc8c181b2fa22bb3` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:47.306975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e4a066d2534edebc8c181b2fa22bb3)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:47.308945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:47.360406 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e4a066d2534edebc8c181b2fa22bb3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.361083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e4a066d2534edebc8c181b2fa22bb3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.373673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e4a066d2534edebc8c181b2fa22bb3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.377235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:47.381510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e4a066d2534edebc8c181b2fa22bb3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.390566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7d711c1b-63e6-4a8b-867c-e4facf785dd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e4a066d2534edebc8c181b2fa22bb3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.396634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1070/2196] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:47 2022] DELETE /identity/v3/users/a4e4a066d2534edebc8c181b2fa22bb3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:47.702854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:47.704395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:47.722485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a87b8-6b32-426a-8ffb-3831ddf07c63 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:47.735572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:47.788454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:47.789127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:47.789923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:47.790192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] PATH_INFO: `/v3/projects/365d9d95597f450d98547a9c4083c2b4` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:47.791300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365d9d95597f450d98547a9c4083c2b4)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:47.793250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:47.839204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365d9d95597f450d98547a9c4083c2b4', 'request_id': 'req-109b2584-74a5-474d-a8c1-66f1f302e4c2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.846229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365d9d95597f450d98547a9c4083c2b4', 'request_id': 'req-109b2584-74a5-474d-a8c1-66f1f302e4c2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.852537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365d9d95597f450d98547a9c4083c2b4', 'request_id': 'req-109b2584-74a5-474d-a8c1-66f1f302e4c2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.863226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] The token cache is being invalidate because project 365d9d95597f450d98547a9c4083c2b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:47.863517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.864173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-109b2584-74a5-474d-a8c1-66f1f302e4c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:47.865920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1127/2197] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:47 2022] DELETE /identity/v3/projects/365d9d95597f450d98547a9c4083c2b4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:47.871608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:47.871925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:47.872478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:47.894573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:47.895851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:47.955483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1071/2198] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:47.964015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:47.964331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:47.964609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:47.984279 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:47.985485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:47.993753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1128/2199] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:47.998060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.007744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ae30a-26e7-403c-bdd0-e8be7b7e92eb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.014272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.015131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.015412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.015713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.016399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:48.016706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.018338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.032117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1072/2200] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:48.036521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.047455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdae8b7-5606-43f7-a203-e54bf4ff12e4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.053947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.054552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.055451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.055718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.056370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.059256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.095176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1129/2201] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:48.096453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.107399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190e6d9-42ef-4420-ac41-4b540b105a10 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.179459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.180672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.180957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.181235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.181874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.183817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.204522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1073/2202] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:48.208654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.219665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0bfa83-12af-49c8-af24-46830e56e99a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.224343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.224958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.225238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.225510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.226147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.228525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.238492 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1130/2203] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:48.242666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.252799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7051dba-a8da-47e7-9288-7319024a67d2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.257463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.258229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.258469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.258955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.259713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.261384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e9ec7-975f-4c61-9f23-0e354226530f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.271950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1074/2204] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:48.276997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.286981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8b7b8-d5dd-4c6a-9419-8e821947eeb3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.292305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.292949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.293232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.293509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] PATH_INFO: `/v3/projects/92bf9ae3ee8445609b5d52e526ce4eef/users/9a7e70c2c8074def812c4a923060957f/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.307902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bf9ae3ee8445609b5d52e526ce4eef, user_id=9a7e70c2c8074def812c4a923060957f, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.310644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.323743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1131/2205] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:48 2022] PUT /identity/v3/projects/92bf9ae3ee8445609b5d52e526ce4eef/users/9a7e70c2c8074def812c4a923060957f/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:48.333918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.334205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.334479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.363059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] MFA Rules not processed for user `9a7e70c2c8074def812c4a923060957f`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:48.364577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:48.408551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:48.432499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1075/2206] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:48.437000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.483143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:48.502616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d36e4-de09-48ec-a4e5-e0d8dce5260d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.507304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.508026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.508531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.508808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.509561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.511459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.538798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1132/2207] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:48.543429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.554365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086793ce-fde8-4817-87a1-31047803ebb1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.610498 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.611100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.611380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.611648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.612309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.614481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.636548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1076/2208] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:48.640671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.651780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606d8d-bbf2-4bcf-ad53-7cfead90617d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.656471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.657087 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.657365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.657633 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.658274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.659912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.671670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1133/2209] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:48.675773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.685776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f04c5c-6dae-47d2-813d-b465622e43a5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.690481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.691149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.691454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.691720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] PATH_INFO: `/v3/projects/ac872e1562554edbb2893f53c7efa2e8/users/c1f8a79f2621418c80a73893fefe1919/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.707159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac872e1562554edbb2893f53c7efa2e8, user_id=c1f8a79f2621418c80a73893fefe1919, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.710224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.725163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1077/2210] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:48 2022] PUT /identity/v3/projects/ac872e1562554edbb2893f53c7efa2e8/users/c1f8a79f2621418c80a73893fefe1919/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:48.729923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.741111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7383e2-af6d-4e94-b85f-ed424d66f74d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.834544 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.835286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.835635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.836106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.836838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.839074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f24b56-aa33-47ab-b99d-5845e109a9af admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.850125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1134/2211] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:48.854680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:48.865735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe48b92-128e-41e2-85ae-63bd2cd0920c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:48.872496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:48.873182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.873465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.873743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] PATH_INFO: `/v3/projects/ac872e1562554edbb2893f53c7efa2e8/users/c1f8a79f2621418c80a73893fefe1919/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.875743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac872e1562554edbb2893f53c7efa2e8, user_id=c1f8a79f2621418c80a73893fefe1919, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:48.878450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:48.891239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1078/2212] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:48 2022] PUT /identity/v3/projects/ac872e1562554edbb2893f53c7efa2e8/users/c1f8a79f2621418c80a73893fefe1919/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:48.903471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:48.903763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:48.904080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:48.948085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] MFA Rules not processed for user `c1f8a79f2621418c80a73893fefe1919`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:48.949322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:49.042333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1135/2213] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:49.053663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:49.056616 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:49.118409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450fb490-6013-480e-b62b-8614bd3478d8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:49.122217 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:49.122822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:49.123096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:49.123369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:49.132764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:49.135285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:49.145821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1079/2214] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:49.272549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:49.274182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:49.286249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aca03f-f09d-433c-b7a3-50d02666a594 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:49.290338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:49.290890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:49.291165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:49.291436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:49.332252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:49.358132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:49.359696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:49.369988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1136/2215] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:53.130948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:53.171961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:53.199517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91aa782-cf0d-459e-a521-61a7c4a926ef service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:53.204667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:53.205301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.205626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.205851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] PATH_INFO: `/v3/users/9a7e70c2c8074def812c4a923060957f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.206987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a7e70c2c8074def812c4a923060957f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:53.209301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:53.209583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:53.211568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.226233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4264b2-28d0-47d6-afdb-6fc47285049f service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:53.231417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:53.232133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.232409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.233021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] PATH_INFO: `/v3/users/e045d5a6d9674a52844c94d9d56c97dd` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.233823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e045d5a6d9674a52844c94d9d56c97dd)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:53.235970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:53.261711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a7e70c2c8074def812c4a923060957f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.262435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a7e70c2c8074def812c4a923060957f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.275741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a7e70c2c8074def812c4a923060957f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.283509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a7e70c2c8074def812c4a923060957f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.290215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e045d5a6d9674a52844c94d9d56c97dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.292219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e045d5a6d9674a52844c94d9d56c97dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.292357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-960cece8-1390-404c-b33e-8a1e833d2dca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a7e70c2c8074def812c4a923060957f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.301519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e045d5a6d9674a52844c94d9d56c97dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.302611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1080/2216] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:53 2022] DELETE /identity/v3/users/9a7e70c2c8074def812c4a923060957f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:53.308430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.308723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.309458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.329974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e045d5a6d9674a52844c94d9d56c97dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.337564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:53.338374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-fb824556-3994-4acf-ab51-8bfcf3f3aacc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e045d5a6d9674a52844c94d9d56c97dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:53.338786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.344887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1137/2217] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:53 2022] DELETE /identity/v3/users/e045d5a6d9674a52844c94d9d56c97dd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:53.350738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.351756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.352068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.379037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:53.380334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.412337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1081/2218] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:53.422105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:53.423943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.442566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1138/2219] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:53.452690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:53.454070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.500731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec10cb-0938-4564-a03e-28f2d25d4153 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:53.518053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:53.518416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:53.523237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920b468-8760-4a0e-b254-a0ad56663981 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:53.530684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:53.531989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.532727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.533615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.546536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:53.548041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:53.576267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1139/2220] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:53.589900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:53.590529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.590811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.591079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.603359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:53.605148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2d2a82-e4fc-47a0-a9f1-4a63c72dbe9c service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:53.621370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1082/2221] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:53.949689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:53.951200 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:53.972450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d73adb-26d0-4bb8-aed4-24f7345230a5 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:53.980500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:53.981078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:53.981475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:53.981773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] PATH_INFO: `/v3/users/b63b0236f8584b9fbdee6f0e52ad64f5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:53.983193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63b0236f8584b9fbdee6f0e52ad64f5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:53.985762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.096987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63b0236f8584b9fbdee6f0e52ad64f5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.097891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63b0236f8584b9fbdee6f0e52ad64f5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.126432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63b0236f8584b9fbdee6f0e52ad64f5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.147678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63b0236f8584b9fbdee6f0e52ad64f5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.165485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63b0236f8584b9fbdee6f0e52ad64f5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.194712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1140/2222] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:53 2022] DELETE /identity/v3/users/b63b0236f8584b9fbdee6f0e52ad64f5 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:54.194808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.194808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.194808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.223416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:54.225235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.322782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1083/2223] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:54.336021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.337280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.410860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.414573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a3dc-dfe9-41da-b67d-7a8858be0b8d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.416556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.418719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.419428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.419781 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.420300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.435277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.436785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.465728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1141/2224] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:54.465975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.467948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.484960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570ea196-f3f1-49b3-bc47-274ac6e1b982 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.485632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026e2a90-4cb5-4014-b614-58d7dab78345 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.494211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.494944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.498199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.498484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] PATH_INFO: `/v3/projects/2ee02dba06c5486ca1b28d2fd75a8636` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.500163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ee02dba06c5486ca1b28d2fd75a8636)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.502736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.502877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.512316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.512751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.513041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] PATH_INFO: `/v3/projects/92bf9ae3ee8445609b5d52e526ce4eef` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.517488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bf9ae3ee8445609b5d52e526ce4eef)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.520584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.588707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ee02dba06c5486ca1b28d2fd75a8636', 'request_id': 'req-5cd93acb-a582-4686-b670-46f341b169e9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.600038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bf9ae3ee8445609b5d52e526ce4eef', 'request_id': 'req-15210675-3726-4de6-9bcd-4097c0a5d620'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.605080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ee02dba06c5486ca1b28d2fd75a8636', 'request_id': 'req-5cd93acb-a582-4686-b670-46f341b169e9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.614583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ee02dba06c5486ca1b28d2fd75a8636', 'request_id': 'req-5cd93acb-a582-4686-b670-46f341b169e9'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.617481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:54.619514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bf9ae3ee8445609b5d52e526ce4eef', 'request_id': 'req-15210675-3726-4de6-9bcd-4097c0a5d620'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.639642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bf9ae3ee8445609b5d52e526ce4eef', 'request_id': 'req-15210675-3726-4de6-9bcd-4097c0a5d620'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.649203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] The token cache is being invalidate because project 2ee02dba06c5486ca1b28d2fd75a8636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:54.650060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.651391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5cd93acb-a582-4686-b670-46f341b169e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.654425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1142/2225] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:54 2022] DELETE /identity/v3/projects/2ee02dba06c5486ca1b28d2fd75a8636 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:54.662624 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d77f5f-497c-44f3-b912-67786de6bc11 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.664472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d77f5f-497c-44f3-b912-67786de6bc11 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.666908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d77f5f-497c-44f3-b912-67786de6bc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.676566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] The token cache is being invalidate because project 92bf9ae3ee8445609b5d52e526ce4eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:54.676907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.678603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.681425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1084/2226] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:54 2022] DELETE /identity/v3/projects/92bf9ae3ee8445609b5d52e526ce4eef => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:54.685061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.686474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.696528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f9d77f5f-497c-44f3-b912-67786de6bc11 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:54.698026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f9d77f5f-497c-44f3-b912-67786de6bc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.752062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:54.804677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15210675-3726-4de6-9bcd-4097c0a5d620 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.811152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.811865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.812159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.812990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] PATH_INFO: `/v3/users/c1f8a79f2621418c80a73893fefe1919` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.813096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1143/2227] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:54.814153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f8a79f2621418c80a73893fefe1919)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.816148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.830049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.831013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.831303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.860421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:54.862146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.873622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1144/2228] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:54.877353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.888142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f8a79f2621418c80a73893fefe1919'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.890486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088938d0-fa72-4e99-ba44-215d24beee5b None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.890784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f8a79f2621418c80a73893fefe1919'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.896646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.897335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.898187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.898890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.899937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:54.900297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.902468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.925214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1145/2229] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:01:54.926231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.928609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:54.932068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f8a79f2621418c80a73893fefe1919'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.947449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f8a79f2621418c80a73893fefe1919'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.948301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256818-e7a6-466c-b077-1588ea8ec043 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:54.954970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:54.955676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:54.955990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:54.956261 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:54.956919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:54.957192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f8a79f2621418c80a73893fefe1919'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:54.958880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:54.966373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1085/2230] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:54 2022] DELETE /identity/v3/users/c1f8a79f2621418c80a73893fefe1919 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:54.967261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:54.969316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:55.018824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1146/2231] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:54 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:55.022792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.067666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c8b7b-1ec2-4c1b-99e6-43d7faac5867 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.131023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d24f50-d144-41fa-aebc-aaed51839e74 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.137692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.138324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.138606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.138885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.139527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.142345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.177549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.178211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.178510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.178763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] PATH_INFO: `/v3/projects/d321dbfe7d6c4600b6b1d40d1e55c655` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.185763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1147/2232] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:55 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:55.186917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d321dbfe7d6c4600b6b1d40d1e55c655)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.189024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.191014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.220414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7fdf4-dba8-4a14-bd29-2cc44303c7d2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.222995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:55.235305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.235975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.236265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.236543 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.237203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.239088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.258393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1148/2233] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:55.261169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.277356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6023c3-0504-4d58-808e-b5d8d39e8a8d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.285289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.286283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.286626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.287032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.288095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.292017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.304724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd321dbfe7d6c4600b6b1d40d1e55c655', 'request_id': 'req-e5884ef6-8d63-48c9-934a-55c995c21f69'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.318133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd321dbfe7d6c4600b6b1d40d1e55c655', 'request_id': 'req-e5884ef6-8d63-48c9-934a-55c995c21f69'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.329804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1149/2234] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:55.331056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.331284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd321dbfe7d6c4600b6b1d40d1e55c655', 'request_id': 'req-e5884ef6-8d63-48c9-934a-55c995c21f69'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.351739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e49fc-a3cb-43bf-9ef1-e9ad36b597e2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.363448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] The token cache is being invalidate because project d321dbfe7d6c4600b6b1d40d1e55c655 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:55.364336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.367324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.370592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1086/2235] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:54 2022] DELETE /identity/v3/projects/d321dbfe7d6c4600b6b1d40d1e55c655 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:55.374819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:55.380779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.384427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:55.503069 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.504120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.504423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.504880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] PATH_INFO: `/v3/projects/11cf864b785b4b638efbc2eeee879cca/users/c0a4336e0f8847e99104307a2f306131/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.520375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5884ef6-8d63-48c9-934a-55c995c21f69 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.532278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=11cf864b785b4b638efbc2eeee879cca, user_id=c0a4336e0f8847e99104307a2f306131, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.533817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:55.534642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.535071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.535495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] PATH_INFO: `/v3/users/b73f4f37c85d49219dc686e23a7da1d9` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.536808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b73f4f37c85d49219dc686e23a7da1d9)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:55.537321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.541761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:55.587853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1150/2236] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:55 2022] PUT /identity/v3/projects/11cf864b785b4b638efbc2eeee879cca/users/c0a4336e0f8847e99104307a2f306131/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:55.588894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:55.598703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:55.658968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b73f4f37c85d49219dc686e23a7da1d9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.660046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b73f4f37c85d49219dc686e23a7da1d9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.692037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b73f4f37c85d49219dc686e23a7da1d9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.702962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b73f4f37c85d49219dc686e23a7da1d9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.718146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-54768b59-b4c3-418f-9ccc-f1afec5d7a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b73f4f37c85d49219dc686e23a7da1d9'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:55.734059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1087/2237] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:55 2022] DELETE /identity/v3/users/b73f4f37c85d49219dc686e23a7da1d9 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:55.735959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:55.745007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:55.745464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:55.792315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac9855-6802-4648-a4b1-b9c3d8260d2b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:55.792526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] MFA Rules not processed for user `c0a4336e0f8847e99104307a2f306131`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:55.794138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:55.799921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:56.002154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:56.003246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.004548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.005032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] PATH_INFO: `/v3/projects/ac872e1562554edbb2893f53c7efa2e8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.006661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac872e1562554edbb2893f53c7efa2e8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:56.016656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:56.049425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1088/2238] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:56.066168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:56.068064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.141065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:56.142076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac872e1562554edbb2893f53c7efa2e8', 'request_id': 'req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.157091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac872e1562554edbb2893f53c7efa2e8', 'request_id': 'req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.167037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac872e1562554edbb2893f53c7efa2e8', 'request_id': 'req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.187210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] The token cache is being invalidate because project ac872e1562554edbb2893f53c7efa2e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:56.187673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.189081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-c35251a1-37fe-44f5-b9e6-23f118ec7eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.191365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1151/2239] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:55 2022] DELETE /identity/v3/projects/ac872e1562554edbb2893f53c7efa2e8 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:56.198115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edc22-994b-44ea-823e-7ba127522f24 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.198750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edc22-994b-44ea-823e-7ba127522f24 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.199296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422edc22-994b-44ea-823e-7ba127522f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.241398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead82726-f727-454a-8d30-fa83cf091247 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:56.245611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-422edc22-994b-44ea-823e-7ba127522f24 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:56.246949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-422edc22-994b-44ea-823e-7ba127522f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.313358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:56.314108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.314467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.314818 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.316785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.368667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1152/2240] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:56.376631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.377074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.377357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.398080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:56.399476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:56.414367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1089/2241] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:56.415376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:56.416833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.420602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:56.422464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.432323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230ccca-02ef-455a-9d3e-59ba2cd9d21a service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:56.434760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1153/2242] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:56.443466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:56.453985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:56.454742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.455033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.455393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] PATH_INFO: `/v3/projects/b0997b80ba0047d48fb86f1ffbf82645` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.457187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0997b80ba0047d48fb86f1ffbf82645)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:56.459475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:56.475415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:56.490179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e31df-c7e1-4b61-ba51-44c51c3268e3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:56.498082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:56.500978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.500978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.500978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.501331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:56.501421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:56.504189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:56.526675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1154/2243] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:56.531757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:56.543958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0997b80ba0047d48fb86f1ffbf82645', 'request_id': 'req-947e2bde-300b-43a5-8fdb-1c986aadb542'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.558743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d3dbe-8ee2-4d15-8df0-973737f4b0b6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:56.559105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0997b80ba0047d48fb86f1ffbf82645', 'request_id': 'req-947e2bde-300b-43a5-8fdb-1c986aadb542'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.574149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0997b80ba0047d48fb86f1ffbf82645', 'request_id': 'req-947e2bde-300b-43a5-8fdb-1c986aadb542'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.604466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] The token cache is being invalidate because project b0997b80ba0047d48fb86f1ffbf82645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:01:56.604972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.607890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-947e2bde-300b-43a5-8fdb-1c986aadb542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:56.613124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1090/2244] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:01:56 2022] DELETE /identity/v3/projects/b0997b80ba0047d48fb86f1ffbf82645 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:56.618755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27969dc-f328-4b7c-b72a-42d473e545a1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.619784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27969dc-f328-4b7c-b72a-42d473e545a1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.620219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27969dc-f328-4b7c-b72a-42d473e545a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.661325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c27969dc-f328-4b7c-b72a-42d473e545a1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:56.663730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c27969dc-f328-4b7c-b72a-42d473e545a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.723642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:56.724467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.724843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.725234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.725980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:56.728232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:56.791431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1155/2245] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/projects => generated 367 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:56.793051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:56.797554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.803193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1091/2246] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:56.816636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:56.820365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:56.820527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:56.875473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:56.877205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:56.948596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:57.007841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcd8404-5d4c-48f7-867c-d9b9f714222a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.021662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.024366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.025125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.025676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.027865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.033417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.036913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1092/2247] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:57.041401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.064139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d81127b-1a3e-432e-bbdc-49fbff07e993 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.083242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.084212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.089447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1156/2248] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:56 2022] POST /identity/v3/users => generated 284 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:57.089585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.089585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.089585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:57.089585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.090796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.090925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.108693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1093/2249] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:57.113335 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.116903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a94a2a-fd24-4915-8ad6-7a1751f8ac10 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.131061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b81aa-baa8-4403-8438-c1ca5cbc201e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.133367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.134067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.134361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.134611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.135270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.136985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.154327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.154959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.155237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.155507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.156169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.158129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.161760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1157/2250] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:57.165668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.185679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ac25f-4bfb-438b-806b-9bda9c3fcc6d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.197423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.198147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.198407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.198679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.199436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.201210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.217782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1158/2251] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:57.223440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1094/2252] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:57.233117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.233355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.242194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:57.248215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9312ef-1d0f-4eb6-9a8c-5d9c163e1519 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.337585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.339047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.339428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.340013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] PATH_INFO: `/v3/projects/46c7335c57254f98b71dd06a4b228208/users/177d8812bd614b58911757f27676a008/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.365040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c7335c57254f98b71dd06a4b228208, user_id=177d8812bd614b58911757f27676a008, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.369780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c130d-7ae6-4f23-ace6-054612737cdc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.388616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1159/2253] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:57 2022] PUT /identity/v3/projects/46c7335c57254f98b71dd06a4b228208/users/177d8812bd614b58911757f27676a008/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:57.399973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.400440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.401158 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.408021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d87fa-658f-425f-821a-5a5d18a91aa2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.532079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] MFA Rules not processed for user `177d8812bd614b58911757f27676a008`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:57.537714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:57.603535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.604139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.604422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.604728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.605388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.609951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.650927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1095/2254] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/users => generated 291 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:57.652682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.666068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d571b432-193d-496f-84a3-b659c0a21e64 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.674443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.675182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.676249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.676739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.677890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.682205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.693139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1160/2255] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:57.694996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.701700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1096/2256] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:57.703477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.708274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6278a1b3-25c3-42e8-9766-e29f87d5c018 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.714993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.716004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.716302 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.716424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5032dc44-c9cb-473c-9e06-ea43d9e67a20 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.716748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.717408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.719552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.721707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.722526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.723230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.723663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.724512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.727311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.740307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1097/2257] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:57.745413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.750845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:57.757916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c468ff8-3727-4137-8986-eae526491503 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.760559 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1161/2258] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:57.764735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.862902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c514eee-2be8-47df-a697-e28a85d49125 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.871278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.872079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.872734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.873109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.874449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.876619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.894022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.894785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.895122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.895504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] PATH_INFO: `/v3/projects/2fdc96f9751d400c9b47f698f8a028e7/users/d0cb41cb929642009f0cd08e7196ec4e/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.910737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1162/2259] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:57.916811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:57.925898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fdc96f9751d400c9b47f698f8a028e7, user_id=d0cb41cb929642009f0cd08e7196ec4e, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.929685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b0f44-39cf-4abe-bec8-a6d6ee12bbd5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.934083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c99c7da-1af7-4fc2-8029-08c45895e0a1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:57.942970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:57.942970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.942970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.943222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.944642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:57.946991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:57.952000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1098/2260] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:57 2022] PUT /identity/v3/projects/2fdc96f9751d400c9b47f698f8a028e7/users/d0cb41cb929642009f0cd08e7196ec4e/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:57.967354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:57.967742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:57.968084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:57.979847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:57.984234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1163/2261] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:57.990586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.010821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb3aa3-159e-44d1-a010-2ac49d05cb2f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.028934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] MFA Rules not processed for user `d0cb41cb929642009f0cd08e7196ec4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:58.030295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:58.093095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.093612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.094049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.094406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] PATH_INFO: `/v3/projects/c2177d56d7374158aff12821ca8de602/users/215660f3ec2a45829a4264ef62877c61/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.112876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1099/2262] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:58.113049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2177d56d7374158aff12821ca8de602, user_id=215660f3ec2a45829a4264ef62877c61, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.114276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.117276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.126520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2c69dd-fdd5-4aa0-96dc-08082d87fc56 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.133291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1164/2263] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:57 2022] PUT /identity/v3/projects/c2177d56d7374158aff12821ca8de602/users/215660f3ec2a45829a4264ef62877c61/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:58.137351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.211643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.213451 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.213841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.214225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.214959 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.217039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.217910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b69941-e7e3-4646-9b74-20d4d446d84a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.223714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.224361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.224641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.224916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.225814 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.227511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.229991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:58.242849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1165/2264] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:01:58.247155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.256101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1100/2265] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:58 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:58.257918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.262749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7e8ccb-315d-44ad-bcbe-bcbfe63392ad admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.337860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.338765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.339362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.339804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] PATH_INFO: `/v3/projects/c2177d56d7374158aff12821ca8de602/users/215660f3ec2a45829a4264ef62877c61/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.343688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2177d56d7374158aff12821ca8de602, user_id=215660f3ec2a45829a4264ef62877c61, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.344650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2830e-cdf5-49d3-b3c5-1f28c4ea24b0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.347116 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900a0a34-6984-465e-b2b2-ab954c2ef5ae admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.349839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.351332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.351697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.352223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.352836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.355004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.363893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1166/2266] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:58 2022] PUT /identity/v3/projects/c2177d56d7374158aff12821ca8de602/users/215660f3ec2a45829a4264ef62877c61/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:58.374341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.374967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.375194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.385823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1101/2267] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:01:58 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:01:58.390214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.401375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a354b-83c2-44c8-98f3-7b53a9cf37cd admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.405619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:58.406909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] MFA Rules not processed for user `215660f3ec2a45829a4264ef62877c61`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:58.408269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:58.475325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.475993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.476272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.477879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.478550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.480204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.491867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1102/2268] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:58.500885 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.516313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c87a6-5d42-4e5d-863d-3b86efb0f3ff admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.517188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1167/2269] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:01:58.542482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.543971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:58.544464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.545144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.545421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.545694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] PATH_INFO: `/v3/projects/f665dba0939d47f98883a9c7dc17660c/users/3c851bb940364643be1cbd5a8c7905b8/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.566013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f665dba0939d47f98883a9c7dc17660c, user_id=3c851bb940364643be1cbd5a8c7905b8, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.569703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.599365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1103/2270] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:58 2022] PUT /identity/v3/projects/f665dba0939d47f98883a9c7dc17660c/users/3c851bb940364643be1cbd5a8c7905b8/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:58.605186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.630304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ce9e60-fd39-4188-9801-4ec9719c745e None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.690964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:58.720643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1cd36-d24e-40a6-8197-850d397ad664 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.731037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.731498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.731802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.732095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.736081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.737125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.740253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.741479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.744795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.746585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.769962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1104/2271] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:01:58.800011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:58.816867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579bbba-29af-4a93-b292-ec6937ac4026 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:58.817790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.819450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b85ac1-9027-43c9-9d49-b1a7d0530791 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.821919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:58.822598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.822881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.823158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] PATH_INFO: `/v3/projects/f665dba0939d47f98883a9c7dc17660c/users/3c851bb940364643be1cbd5a8c7905b8/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.825202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f665dba0939d47f98883a9c7dc17660c, user_id=3c851bb940364643be1cbd5a8c7905b8, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:58.828426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:58.834526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1168/2272] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:58 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:58.848195 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1105/2273] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:01:58 2022] PUT /identity/v3/projects/f665dba0939d47f98883a9c7dc17660c/users/3c851bb940364643be1cbd5a8c7905b8/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:01:58.855171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:58.855765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:58.856249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:58.895477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] MFA Rules not processed for user `3c851bb940364643be1cbd5a8c7905b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:58.896612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:58.960867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.009381 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1169/2274] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:01:58 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:59.023431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.025694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:59.045767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dfee06-00bd-4ea0-a1e8-beb983327f86 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.049834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.050532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.050943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.051964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.084191 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:59.125475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6dec2-7071-49fb-8c19-235fe9da55c3 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.130633 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.131141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.132067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.132351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.145380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.145577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.147774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85a6ee-47c5-4c06-ac5d-44bb61561069 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.149121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de28cf-6e57-40b0-8f46-da507322421f service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.160595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1170/2275] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:01:59.167549 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1106/2276] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:59.591905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.592432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.592741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.594606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1171/2277] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:01:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:01:59.604744 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.605344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.605344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.618798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.621147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:59.649808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:01:59.656923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:01:59.659086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:59.735771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1107/2278] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:01:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:01:59.736550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.736754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930be3-f39c-4822-bbd6-c6cf2bfbbb9f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.742718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.743717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.744101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.744447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] PATH_INFO: `/v3/users/88dab5446aff4214b3c15c7bb1d3b79c` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.745785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88dab5446aff4214b3c15c7bb1d3b79c)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.747838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.748479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6928f40d-73f3-4f9e-b83e-3830dbb81166 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.754984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c851bb940364643be1cbd5a8c7905b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f665dba0939d47f98883a9c7dc17660c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.754984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.755233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.755504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.757433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:01:59.757433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.758887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.774753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1108/2279] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:01:59.774753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.774753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:59.792398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75f8de-ddb8-4fdc-8a3b-c15727c67883 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.798040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.798662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.798950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.799225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.808046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.809691 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.809860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88dab5446aff4214b3c15c7bb1d3b79c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:59.810631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88dab5446aff4214b3c15c7bb1d3b79c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:59.819876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88dab5446aff4214b3c15c7bb1d3b79c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:59.820726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1109/2280] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:01:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:01:59.821539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.831975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ec2cf-bc22-4627-b221-52bf4a55e678 service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.833919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88dab5446aff4214b3c15c7bb1d3b79c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:59.838250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c851bb940364643be1cbd5a8c7905b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f665dba0939d47f98883a9c7dc17660c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:01:59.838748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:01:59.839041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:01:59.839321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:01:59.839988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:01:59.842011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:01:59.843063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88dab5446aff4214b3c15c7bb1d3b79c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:01:59.851120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1172/2281] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:01:59 2022] DELETE /identity/v3/users/88dab5446aff4214b3c15c7bb1d3b79c => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:01:59.886370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1110/2282] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:01:59 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:01:59.887638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:01:59.889374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:01:59.963500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b57f2-5ae9-48ae-a830-348c9134fd04 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:01:59.999229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:00.023496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c851bb940364643be1cbd5a8c7905b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f665dba0939d47f98883a9c7dc17660c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.024136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.024417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.024696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.025374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:00.025682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.027298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.039614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1173/2283] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:00.045526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.047912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.099937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.110385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c4286-7a72-412e-828b-d7cfb793d286 tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.115197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3c851bb940364643be1cbd5a8c7905b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f665dba0939d47f98883a9c7dc17660c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.116459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.116795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.117236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] PATH_INFO: `/v3/projects/44f98c512d3f4292ae456241f8685cd0` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.118362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorizing `identity:delete_project(project_id=44f98c512d3f4292ae456241f8685cd0)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.118989 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c909c19-9e35-4206-a82a-83f15de4c47d tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.120297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.123708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.124365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.124646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.124920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] PATH_INFO: `/v3/projects/588cfe4bc032419f901b4989248d9a0f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.126034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588cfe4bc032419f901b4989248d9a0f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.127991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.136025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:00.175034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44f98c512d3f4292ae456241f8685cd0', 'request_id': 'req-cfc7cf15-e778-476a-9267-8279cd2d2e8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.183360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44f98c512d3f4292ae456241f8685cd0', 'request_id': 'req-cfc7cf15-e778-476a-9267-8279cd2d2e8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.184462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588cfe4bc032419f901b4989248d9a0f', 'request_id': 'req-60cca802-e414-4011-a446-c752772964e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.190857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44f98c512d3f4292ae456241f8685cd0', 'request_id': 'req-cfc7cf15-e778-476a-9267-8279cd2d2e8c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.191947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588cfe4bc032419f901b4989248d9a0f', 'request_id': 'req-60cca802-e414-4011-a446-c752772964e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.199263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588cfe4bc032419f901b4989248d9a0f', 'request_id': 'req-60cca802-e414-4011-a446-c752772964e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.204115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] The token cache is being invalidate because project 44f98c512d3f4292ae456241f8685cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:00.204403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.205067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfc7cf15-e778-476a-9267-8279cd2d2e8c tempest-VolumeQuotaClassesTest-309984258 tempest-VolumeQuotaClassesTest-309984258-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.207174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1174/2284] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:00 2022] DELETE /identity/v3/projects/44f98c512d3f4292ae456241f8685cd0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:00.211848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] The token cache is being invalidate because project 588cfe4bc032419f901b4989248d9a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:00.212147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.212813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-60cca802-e414-4011-a446-c752772964e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.214511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1111/2285] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:00 2022] DELETE /identity/v3/projects/588cfe4bc032419f901b4989248d9a0f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:00.219214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.219584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.219758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.241614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:00.242776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.308369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1175/2286] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:00.316588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.316874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.317147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.338720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:00.339974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.351485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1112/2287] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:00.353220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.363055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee01030-3a46-462f-a541-e9bdc74afc32 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.369285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.369556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.369830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.370100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.370750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:00.371048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.372668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.387906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1176/2288] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:02:00.389936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.402067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ede5-9e3d-4759-9dbe-9d16e77f3c6c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.407459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.408165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.408446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.408716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.409350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.411232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.436488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.438378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.445223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1113/2289] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:00.446189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.452993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850dd54-57cc-4f09-9431-ad961422349e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.498125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:00.513395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8fe2a-8e17-4240-ab33-3438efb6bbea admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.516318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.517031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.517382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.517722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] PATH_INFO: `/v3/users/d0cb41cb929642009f0cd08e7196ec4e` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.518030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.518626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.518903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.519147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0cb41cb929642009f0cd08e7196ec4e)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.519258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.519775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.521305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.521717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.545349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1114/2290] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:00.549586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.560883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d40773f-307d-4594-80b8-58def236395a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.566416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.568099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.570214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.570918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.571585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.574198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.595877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0cb41cb929642009f0cd08e7196ec4e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.597073 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0cb41cb929642009f0cd08e7196ec4e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.604806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1115/2291] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:02:00.608895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.609951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0cb41cb929642009f0cd08e7196ec4e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.614283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.625958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0cb41cb929642009f0cd08e7196ec4e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.628514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b72e207-7d4f-402f-9797-f9030f6faa51 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.634000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1d7b6efb-566b-49e7-b263-24738c87215d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0cb41cb929642009f0cd08e7196ec4e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:00.634592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.635204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.635482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.635756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.636447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.638041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.643945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1177/2292] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:00 2022] DELETE /identity/v3/users/d0cb41cb929642009f0cd08e7196ec4e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:00.645733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.646101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.646446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.653553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1116/2293] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:00.654792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.665887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f6e23-f59b-420d-82be-3681aee0c456 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.667928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:00.669138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.697831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:00.729799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.730482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.730587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1178/2294] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:00.730941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.731275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] PATH_INFO: `/v3/projects/27be5464dad640c28245de15564dd256/users/cb8db3b650f24566b537fcb10e7f46ff/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.741320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.742712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.748647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] RBAC: Authorizing `identity:create_grant(project_id=27be5464dad640c28245de15564dd256, user_id=cb8db3b650f24566b537fcb10e7f46ff, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.751587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fc17fe-5605-4812-afaf-1b5ef3af914f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.765276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1117/2295] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:00 2022] PUT /identity/v3/projects/27be5464dad640c28245de15564dd256/users/cb8db3b650f24566b537fcb10e7f46ff/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:00.770079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.770368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.770644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.789712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:00.790930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:00.801799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096523b-ac6a-4c57-92d9-2f11c61a3f41 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.814189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:00.852249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1118/2296] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:00.853923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.854536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.854810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.855196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.861878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:00.865179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.867256 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.873527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9cb80-48f1-4ddd-ba3f-2613f9fd8dd0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:00.877035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1179/2297] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:00.880694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:00.881434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:00.881765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:00.882030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:00.896111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:00.897966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:00.909221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1119/2298] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:01.257461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:01.259233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:01.274206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0e3f7-e80b-4b57-9e1d-1801f30954ff service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:01.280340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:01.281685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:01.282027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:01.282458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] PATH_INFO: `/v3/projects/2fdc96f9751d400c9b47f698f8a028e7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:01.284399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fdc96f9751d400c9b47f698f8a028e7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:01.286181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:01.346423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fdc96f9751d400c9b47f698f8a028e7', 'request_id': 'req-72f00360-51d6-4ee7-b987-91f2dff1c444'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.358322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fdc96f9751d400c9b47f698f8a028e7', 'request_id': 'req-72f00360-51d6-4ee7-b987-91f2dff1c444'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.368539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fdc96f9751d400c9b47f698f8a028e7', 'request_id': 'req-72f00360-51d6-4ee7-b987-91f2dff1c444'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.385780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] The token cache is being invalidate because project 2fdc96f9751d400c9b47f698f8a028e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:01.386182 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.386986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.388929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1180/2299] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:01 2022] DELETE /identity/v3/projects/2fdc96f9751d400c9b47f698f8a028e7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:01.392773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:01.403671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55d3c5-8332-430c-9839-ed3e25097cb3 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:01.526328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:01.526936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:01.527250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:01.527522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] PATH_INFO: `/v3/users/3c851bb940364643be1cbd5a8c7905b8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:01.529282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c851bb940364643be1cbd5a8c7905b8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:01.531578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:01.594760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:01.596573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c851bb940364643be1cbd5a8c7905b8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.597624 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c851bb940364643be1cbd5a8c7905b8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.612746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c851bb940364643be1cbd5a8c7905b8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.620629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c851bb940364643be1cbd5a8c7905b8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.628245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c21b9bb4-a932-425e-acba-28eea2018e0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c851bb940364643be1cbd5a8c7905b8'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:01.637019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1120/2300] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:01 2022] DELETE /identity/v3/users/3c851bb940364643be1cbd5a8c7905b8 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:01.982446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:01.984353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:01.997539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f00360-51d6-4ee7-b987-91f2dff1c444 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.101678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.102333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.102612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.102883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] PATH_INFO: `/v3/projects/f665dba0939d47f98883a9c7dc17660c` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.104060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] RBAC: Authorizing `identity:delete_project(project_id=f665dba0939d47f98883a9c7dc17660c)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.105983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.169184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:02.176742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f665dba0939d47f98883a9c7dc17660c', 'request_id': 'req-9c20f6e3-d4a5-4526-bebc-28f82007e2af'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:02.193539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f665dba0939d47f98883a9c7dc17660c', 'request_id': 'req-9c20f6e3-d4a5-4526-bebc-28f82007e2af'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:02.201630 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f665dba0939d47f98883a9c7dc17660c', 'request_id': 'req-9c20f6e3-d4a5-4526-bebc-28f82007e2af'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:02.215602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] The token cache is being invalidate because project f665dba0939d47f98883a9c7dc17660c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:02.216090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:02.216861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c20f6e3-d4a5-4526-bebc-28f82007e2af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:02.222632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1181/2301] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:01 2022] DELETE /identity/v3/projects/f665dba0939d47f98883a9c7dc17660c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:02.224496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f52c-ff03-4ff6-bc9b-96857d1911fd None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.225343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f52c-ff03-4ff6-bc9b-96857d1911fd None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.225681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f52c-ff03-4ff6-bc9b-96857d1911fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.254197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1788f52c-ff03-4ff6-bc9b-96857d1911fd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:02.255506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1788f52c-ff03-4ff6-bc9b-96857d1911fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:02.334707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1121/2302] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:02.342623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.342910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.343207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.371944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:02.372365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.372570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.372841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.373540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:02.388961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1182/2303] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:02.390090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.401278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d494cf3-89bd-4680-8247-aa4da1ecb47f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.404054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:02.405347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:02.407487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.408364 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.409020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.409397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.410213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:02.424924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.429690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.461922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1183/2304] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:02.468483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.492125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf91fd-d81f-4f95-92fc-4e2866f7f88b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.521274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1122/2305] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:02.527583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.528239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.528524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.528796 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.529434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.542195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.591886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.594013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:02.620103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1184/2306] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:02 2022] POST /identity/v3/projects => generated 399 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:02.621858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.746955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e78d8f-c719-4c82-b4ce-3351a1580626 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.757505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.758247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.758625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.759010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.760017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.770019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.777585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07624759-cdc6-45c4-9c14-04350f17fa3c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.789279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:02.816336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1185/2307] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:02 2022] POST /identity/v3/users => generated 300 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:02.819460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.834016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f54946-6a35-4c43-a169-6ed900be3b48 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.841761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.842618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.843025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.843662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.844547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.847090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.877833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1186/2308] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:02:02.885672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:02.945688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14a05eb-a3d8-4984-95b8-b973ebf8223c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:02.952241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.953508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.955202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.955202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.964733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:02.967907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:02.967907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:02.968494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:02.969319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:02.972613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:02.998233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1187/2309] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:02 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:03.008937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.024426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b7c23-acbf-4b66-b6e6-e8d9441ca6a4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.034957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:03.036330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.039325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.039726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] PATH_INFO: `/v3/projects/2ce1e37e54a6443c9c0e5518504d3d1a/users/8bce59f5d19a4c468e8fa2e94307b855/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.074168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce1e37e54a6443c9c0e5518504d3d1a, user_id=8bce59f5d19a4c468e8fa2e94307b855, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.077121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e3e12-b370-4300-826f-a1745319c2ab admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.092323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.099022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.114110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1188/2310] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:03 2022] PUT /identity/v3/projects/2ce1e37e54a6443c9c0e5518504d3d1a/users/8bce59f5d19a4c468e8fa2e94307b855/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:03.125225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.125523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.125927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.129213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1123/2311] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:02:03.204453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] MFA Rules not processed for user `8bce59f5d19a4c468e8fa2e94307b855`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:03.206572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:03.245999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:03.304596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1189/2312] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:03 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:03.309420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.326505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787c091-ecea-4fb3-8a5c-e2b406b1f09f service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.439929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:03.440798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.441452 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.441827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.445980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.445980 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.484173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1124/2313] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:03 2022] POST /identity/v3/projects => generated 399 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:03.485276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.498600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951b7c6-f670-43bb-a1e4-96b9bfdf6399 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.610523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:03.611972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.612461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.613072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.613806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.616586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.675826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1190/2314] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:03 2022] POST /identity/v3/users => generated 306 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:03.677502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.697969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893f4bf-ef25-45af-be43-3521edaf8285 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.712237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:03.712930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.713600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.713600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.714140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.715817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.751153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1125/2315] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:03.752043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.772233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7b8080-555f-4fae-b71b-65d0ec77cc54 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.778860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:03.779555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:03.779930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:03.780216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] PATH_INFO: `/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19/users/25b4e267a562445c931492f1e3a3108f/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:03.806973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7aa2b00c9e4c97bafd13328a3cdf19, user_id=25b4e267a562445c931492f1e3a3108f, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:03.809862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:03.871691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1191/2316] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:03 2022] PUT /identity/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19/users/25b4e267a562445c931492f1e3a3108f/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:02:03.875582 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:03.889355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd2461-a95e-4935-9c88-1f6972fe846d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:03.951269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:04.040782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:04.041566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.041941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.042342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.043183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:04.045540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515cbda5-e980-4d61-baa9-c2f91ae81e81 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:04.063924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1126/2317] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:04.073867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:04.087534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424cdd-7049-414b-ab99-8339d085cb80 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:04.111264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:04.111980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.112264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.112539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] PATH_INFO: `/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19/users/25b4e267a562445c931492f1e3a3108f/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.125096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7aa2b00c9e4c97bafd13328a3cdf19, user_id=25b4e267a562445c931492f1e3a3108f, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:04.128086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cae6ed-5961-4eac-9085-ad86ac7588f7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:04.168615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1192/2318] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:04 2022] PUT /identity/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19/users/25b4e267a562445c931492f1e3a3108f/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:04.172493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baf0f7-9dc4-41b4-afad-dd33b9ca2aa7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.172951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baf0f7-9dc4-41b4-afad-dd33b9ca2aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.173571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58baf0f7-9dc4-41b4-afad-dd33b9ca2aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.212392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e529-3366-474a-8875-7c23b4a5ac38 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.212690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e529-3366-474a-8875-7c23b4a5ac38 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.217413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073e529-3366-474a-8875-7c23b4a5ac38 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.221072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1193/2319] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:02:04 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:04.231940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.232282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.232565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.260094 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-58baf0f7-9dc4-41b4-afad-dd33b9ca2aa7 None None] MFA Rules not processed for user `25b4e267a562445c931492f1e3a3108f`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:04.261368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-58baf0f7-9dc4-41b4-afad-dd33b9ca2aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:04.276628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] MFA Rules not processed for user `b0a88b1bdce5483c95b59921dbfc5660`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:04.279495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:04.441694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1127/2320] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:04 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:04.445458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.450151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.450440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.453435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1194/2321] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:02:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:04.485422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:04.486841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:04.495530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:04.500096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:04.503926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507964d4-053d-4a43-9378-17e0aba94769 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:04.512058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:04.512832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.513415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.513538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.526462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:04.528445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:04.537642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:04.559061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1195/2322] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:04.561243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:04.566271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:04.643475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1128/2323] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:02:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:04.664281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:04.687863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b70d4-d4e5-442a-bb72-fb8de9356154 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:04.693811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:04.694737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.695217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.696228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.712147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:04.714777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:04.731900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1196/2324] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:04 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:04.791510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:04.848783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b86902-2829-4e67-8bcd-86abdac7eb6d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:04.873687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:04.875364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:04.875667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:04.876518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:04.911288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:04.915070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:04.938775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1129/2325] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:02:04.967819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:04.968303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:05.005315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc5776-26b5-47b5-b1b2-622fd3c81be5 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:05.043368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:05.044255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:05.044588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:05.045098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:05.213902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:05.215464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232a9f0-a76e-4a0b-8109-b3db9269bd9b service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:05.239524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1197/2326] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:05.454353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:05.465236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:05.471649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:05.471902 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:05.472187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:05.521879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:05.525213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:05.549263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1198/2327] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:05.569134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:05.593536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311fa67-acea-4f1d-82fc-b3298f190bde None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:05.617711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:05.618077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:05.618924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:05.619433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:05.664691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:05.667400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217593ba-5f8a-4d07-848d-671786852a37 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:05.668337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:05.678622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:05.681078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:05.681606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:05.682152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] PATH_INFO: `/v3/users/863ac1e3894449f68cf311f6a2995537` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:05.686133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=863ac1e3894449f68cf311f6a2995537)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:05.689821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:05.705160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1199/2328] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:02:05.824289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '863ac1e3894449f68cf311f6a2995537'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:05.826503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '863ac1e3894449f68cf311f6a2995537'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:05.844159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '863ac1e3894449f68cf311f6a2995537'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:05.861059 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '863ac1e3894449f68cf311f6a2995537'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:05.876209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a5212c14-d903-4fc6-b207-9630fc40bf67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '863ac1e3894449f68cf311f6a2995537'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:05.886877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1130/2329] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:05 2022] DELETE /identity/v3/users/863ac1e3894449f68cf311f6a2995537 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:06.260133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:06.277206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b3f2c-ae91-45e2-84af-7a663944356f service swift] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:06.416084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:06.417491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:06.417923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:06.419477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] PATH_INFO: `/v3/projects/c12e517411b94021b245fb89b6daed29` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:06.421076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12e517411b94021b245fb89b6daed29)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:06.423235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:06.501694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:06.545899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12e517411b94021b245fb89b6daed29', 'request_id': 'req-ebea0b41-4718-467c-965e-e52f4bf68989'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.556749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12e517411b94021b245fb89b6daed29', 'request_id': 'req-ebea0b41-4718-467c-965e-e52f4bf68989'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.557627 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:06.557920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:06.558194 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:06.572040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12e517411b94021b245fb89b6daed29', 'request_id': 'req-ebea0b41-4718-467c-965e-e52f4bf68989'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.594502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:06.595856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:06.602197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] The token cache is being invalidate because project c12e517411b94021b245fb89b6daed29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:06.602705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.603645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.606192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1200/2330] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:06 2022] DELETE /identity/v3/projects/c12e517411b94021b245fb89b6daed29 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:06.609946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:06.612259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:06.730961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1131/2331] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:02:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:06.748985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:06.750907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:06.762081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0b41-4718-467c-965e-e52f4bf68989 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:06.768294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:06.769403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:06.770212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:06.770494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:06.770768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] PATH_INFO: `/v3/users/58a3ed135f254961ab72b5ac6cb1f059` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:06.773190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a3ed135f254961ab72b5ac6cb1f059)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:06.776562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:06.866078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a3ed135f254961ab72b5ac6cb1f059'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.866828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a3ed135f254961ab72b5ac6cb1f059'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.889072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a3ed135f254961ab72b5ac6cb1f059'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.900812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20df75a4-d513-4434-80f5-eccafd434696 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:06.909130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:06.909993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a3ed135f254961ab72b5ac6cb1f059'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.910282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:06.910696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:06.911114 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:06.914226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:06.921923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a3ed135f254961ab72b5ac6cb1f059'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:06.930682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1201/2332] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:06 2022] DELETE /identity/v3/users/58a3ed135f254961ab72b5ac6cb1f059 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:06.931244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:06.933126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b84c2-a56b-40ea-b610-faa32e90eff5 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:06.953469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1132/2333] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:02:07.195483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:07.197882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:07.295374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:07.349710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0831e5-cbb7-460b-93dc-9f6b0128b024 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:07.359553 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:07.360347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:07.360622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:07.360889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] PATH_INFO: `/v3/projects/6bcab14060e94cd6b236026e9814749a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:07.362346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bcab14060e94cd6b236026e9814749a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:07.364698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:07.444772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bcab14060e94cd6b236026e9814749a', 'request_id': 'req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:07.460992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bcab14060e94cd6b236026e9814749a', 'request_id': 'req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:07.469923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bcab14060e94cd6b236026e9814749a', 'request_id': 'req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:07.496130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] The token cache is being invalidate because project 6bcab14060e94cd6b236026e9814749a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:07.496670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:07.497723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad4a5f3f-33df-4bac-9747-0b9ea12b972c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:07.499556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1202/2334] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:07 2022] DELETE /identity/v3/projects/6bcab14060e94cd6b236026e9814749a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:07.505570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:07.505570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:07.505570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:07.550214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:07.555243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:07.785526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1133/2335] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:07.795676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:07.796676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:07.797175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:07.831493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:07.833213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:07.855688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:07.859998 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1203/2336] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:07.868396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b45e76d-258b-40e1-a831-793ef9ecd267 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:07.874170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:07.874909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:07.875291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:07.875661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:07.876463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:07.876865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:07.878728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:07.907501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:07.920874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1134/2337] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:02:07.924688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d20d75-b385-4ef9-aa7e-0e36d4eac94f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:07.930665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:07.931433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:07.931718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:07.932040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:07.932676 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:07.934577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dc144-5c48-47fe-9a1c-15f7588343eb admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:07.988822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1204/2338] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:07 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:07.993634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.013223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8d39c-c5a3-4949-a0e4-3735fd2b4fd7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.048793 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a868ac-5148-4454-8442-4178a4fc0104 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.049090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a868ac-5148-4454-8442-4178a4fc0104 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.050246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a868ac-5148-4454-8442-4178a4fc0104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.093482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-48a868ac-5148-4454-8442-4178a4fc0104 None None] MFA Rules not processed for user `cb8db3b650f24566b537fcb10e7f46ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:08.095000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-48a868ac-5148-4454-8442-4178a4fc0104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:08.176188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.177115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.178309 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.178790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.179805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.182047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:08.215096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1135/2339] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:07 2022] POST /identity/v3/users => generated 289 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:08.216933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.236748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea230d2-3199-4035-a84f-626b1f627125 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.257223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.259173 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.259267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.259538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.262305 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.267442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:08.282015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1205/2340] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:08.301022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.303454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.304170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.307695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1136/2341] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:08.313855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.324386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:08.340983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94209a3b-fe68-45df-a9da-d85bcceed13b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.343837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] MFA Rules not processed for user `cb8db3b650f24566b537fcb10e7f46ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:08.345750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:08.357525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.358306 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.358609 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1206/2342] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:08.363030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.363472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.364296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.366942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:08.375208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.378987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:08.386822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1137/2343] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:08.395754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.417874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:08.429322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1e473-4b6f-485f-9abe-fc002484618a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.441844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.442649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.443154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.450770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] PATH_INFO: `/v3/projects/2375176960a245cea212397a4a5babb0/users/613c09ab1ac94b17bd81e855c09a51ea/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.502486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2375176960a245cea212397a4a5babb0, user_id=613c09ab1ac94b17bd81e855c09a51ea, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.505721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55454e31-a73a-4a9b-8104-e7fb045c8d7d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:08.539431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1138/2344] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:08 2022] PUT /identity/v3/projects/2375176960a245cea212397a4a5babb0/users/613c09ab1ac94b17bd81e855c09a51ea/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:08.552106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.553385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.553815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.606686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5fa16-b09b-4e40-ae19-2e14325f3a2d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.628298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] MFA Rules not processed for user `613c09ab1ac94b17bd81e855c09a51ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:08.629372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:08.803730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.804520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.804901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.805307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.809070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1139/2345] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:08.871807 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:08.874142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:08.918378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.921520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:08.949310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1207/2346] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:08.970477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df833a-abc6-4f5d-b954-14cd5bc46bb0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:08.974750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:08.975471 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:08.975853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:08.976257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:08.992115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:08.993726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a443e763-29ed-4625-9107-b9477cad54a8 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:09.016185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1140/2347] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:09.304987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:09.306683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:09.379465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f5ee-a614-4997-88f5-6b923f4e8e1c None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:09.379924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f5ee-a614-4997-88f5-6b923f4e8e1c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:09.380358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50f5ee-a614-4997-88f5-6b923f4e8e1c None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:09.382320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1141/2348] 10.222.0.78 () {54 vars in 912 bytes} [Wed Aug 24 10:02:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:02:09.388380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:09.388771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:09.389168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:09.431268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:09.449728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:09.452909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:09.467468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e2fb-6c4d-49c7-ad2c-81b336406214 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:09.491066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:09.491698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:09.492027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:09.492305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] PATH_INFO: `/v3/users/177d8812bd614b58911757f27676a008` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:09.510395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=177d8812bd614b58911757f27676a008)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:09.521719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:09.633551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1142/2349] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:09.649453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:09.653570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:09.677984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '177d8812bd614b58911757f27676a008'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:09.679091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '177d8812bd614b58911757f27676a008'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:09.687663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b624d65-d98f-4aee-97cd-b901151a39e3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:09.694338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '177d8812bd614b58911757f27676a008'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:09.697361 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:09.698127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:09.700127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:09.700423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:09.704413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:09.711698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '177d8812bd614b58911757f27676a008'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:09.720066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:09.723082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:09.727212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c9d81-8f4d-4628-8e2a-96e2b91a1140 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:09.734330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '177d8812bd614b58911757f27676a008'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:09.744037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1143/2351] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:09.765943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb940-806b-4c8d-8fce-388f083f9123 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:09.766180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1208/2351] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:09 2022] DELETE /identity/v3/users/177d8812bd614b58911757f27676a008 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:09.767749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb940-806b-4c8d-8fce-388f083f9123 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:09.768996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2cb940-806b-4c8d-8fce-388f083f9123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:09.821169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4b2cb940-806b-4c8d-8fce-388f083f9123 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:09.822718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b2cb940-806b-4c8d-8fce-388f083f9123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:09.968412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1144/2352] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:09.990985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:09.992853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:10.080835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c22a67-64d2-4c22-bb09-61ce6d1db4f5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:10.086749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:10.087927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:10.088413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:10.088922 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:10.099304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:10.101549 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:10.117247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1209/2353] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:10.370100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:10.370895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:10.371316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:10.399288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] MFA Rules not processed for user `cb8db3b650f24566b537fcb10e7f46ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:10.401518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:10.495699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1145/2354] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:10 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:10.550230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:10.555666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:10.627165 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906526d-e6b8-42e4-a269-74a057b069f4 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:10.634147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:10.635100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:10.635908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:10.636487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] PATH_INFO: `/v3/projects/46c7335c57254f98b71dd06a4b228208` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:10.639178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c7335c57254f98b71dd06a4b228208)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:10.642626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:10.744808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c7335c57254f98b71dd06a4b228208', 'request_id': 'req-666ad8cd-5f0c-45d8-b939-83a7f519a59f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:10.756917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c7335c57254f98b71dd06a4b228208', 'request_id': 'req-666ad8cd-5f0c-45d8-b939-83a7f519a59f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:10.769729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c7335c57254f98b71dd06a4b228208', 'request_id': 'req-666ad8cd-5f0c-45d8-b939-83a7f519a59f'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:10.784984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:10.819707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] The token cache is being invalidate because project 46c7335c57254f98b71dd06a4b228208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:10.820035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:10.821812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:10.829025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1210/2355] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:10 2022] DELETE /identity/v3/projects/46c7335c57254f98b71dd06a4b228208 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:10.831018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:10.857988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985719-0949-406a-b368-8601f365c2f8 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:10.912409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:10.914212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:11.076452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666ad8cd-5f0c-45d8-b939-83a7f519a59f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:11.080947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:11.082160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:11.082443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:11.082712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] PATH_INFO: `/v3/users/215660f3ec2a45829a4264ef62877c61` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:11.084871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215660f3ec2a45829a4264ef62877c61)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:11.088126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:11.118024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:11.121949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:11.122442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:11.122910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] PATH_INFO: `/v3/users/5b7ce2127ad2493db1876329194e0333` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:11.125275 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7ce2127ad2493db1876329194e0333)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:11.129315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:11.268224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7ce2127ad2493db1876329194e0333'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.268224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7ce2127ad2493db1876329194e0333'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.288737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:11.291995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7ce2127ad2493db1876329194e0333'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.293597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215660f3ec2a45829a4264ef62877c61'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.294197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215660f3ec2a45829a4264ef62877c61'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.336396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7ce2127ad2493db1876329194e0333'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.367957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7ce2127ad2493db1876329194e0333'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.374100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215660f3ec2a45829a4264ef62877c61'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.388025 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215660f3ec2a45829a4264ef62877c61'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.395417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1211/2356] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:10 2022] DELETE /identity/v3/users/5b7ce2127ad2493db1876329194e0333 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:02:11.398602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:11.401178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-d911d622-0f03-42a3-a1c4-5a91cbf1b4a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215660f3ec2a45829a4264ef62877c61'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.401799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:11.413828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1146/2357] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:10 2022] DELETE /identity/v3/users/215660f3ec2a45829a4264ef62877c61 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:11.415867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:11.416269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:11.416931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:11.465175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:11.467432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:11.570631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563090f-53e4-458a-8517-0deb164bbdf6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:11.579318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1147/2358] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:11.587281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:11.588557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:11.589040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:11.589842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] PATH_INFO: `/v3/users/c0a4336e0f8847e99104307a2f306131` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:11.591523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a4336e0f8847e99104307a2f306131)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:11.594925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:11.595333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:11.597597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:11.628313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:11.708631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a4336e0f8847e99104307a2f306131'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.710177 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a4336e0f8847e99104307a2f306131'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.721949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a4336e0f8847e99104307a2f306131'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.792195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a4336e0f8847e99104307a2f306131'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.809844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f805fab7-1ead-4ff0-b234-e8968f062421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a4336e0f8847e99104307a2f306131'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:11.843341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1212/2359] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:11 2022] DELETE /identity/v3/users/c0a4336e0f8847e99104307a2f306131 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:11.844805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:11.845097 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:11.845378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:11.902357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753dbd0-924f-46e7-b695-e45091fe97b4 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:11.905597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:11.907038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.036143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1213/2360] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:12.036143 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.039904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.060336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86102bfe-3e34-47dd-a4b0-63a88518b7c0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.065536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.066357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.066696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.067035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] PATH_INFO: `/v3/projects/c2177d56d7374158aff12821ca8de602` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.070822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2177d56d7374158aff12821ca8de602)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.074613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.086478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.087465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.088176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.088597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.103342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.128214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.132085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.149900 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1148/2361] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:12.151690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.153387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.169190 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41839fb2-e124-40e5-9546-63c793662cef service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.174472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.175310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.175880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.176351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.193184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.195634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b282e-289f-4d5d-8151-598c68928c53 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.203783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2177d56d7374158aff12821ca8de602', 'request_id': 'req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.219827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1149/2362] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:12.224664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2177d56d7374158aff12821ca8de602', 'request_id': 'req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.238025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2177d56d7374158aff12821ca8de602', 'request_id': 'req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.253569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] The token cache is being invalidate because project c2177d56d7374158aff12821ca8de602 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:12.254509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.255503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e2fc06f6-0e2a-4d41-b78b-16defe8a3718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.258414 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1214/2363] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:12 2022] DELETE /identity/v3/projects/c2177d56d7374158aff12821ca8de602 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:12.263286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.263597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.263967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.283661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:12.294340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:12.299122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.438264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.438623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1150/2364] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:12.439432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.439723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.471436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.471436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.492421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06868d-f24f-49a8-be8f-c923ebca494b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.510010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:12.511666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.513897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.515368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.515736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.516030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] PATH_INFO: `/v3/projects/ac770ae96a814f5cae8e60ba10ca9011` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.517891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac770ae96a814f5cae8e60ba10ca9011)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.519997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.527843 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1215/2365] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:12.531388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.583527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a83bd62-2d44-4da6-b37c-b9d8e5d225b5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.595923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.596431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.596717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.596992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.597659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:12.597953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.600135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.619913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:12.638393 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1216/2366] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:12.643362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.656473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872008c-c801-4746-9ca5-c1415bc31762 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.663606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac770ae96a814f5cae8e60ba10ca9011', 'request_id': 'req-3e8091c7-7ba2-4088-9120-64cd04c9e656'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.675891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac770ae96a814f5cae8e60ba10ca9011', 'request_id': 'req-3e8091c7-7ba2-4088-9120-64cd04c9e656'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.686015 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac770ae96a814f5cae8e60ba10ca9011', 'request_id': 'req-3e8091c7-7ba2-4088-9120-64cd04c9e656'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.700817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] The token cache is being invalidate because project ac770ae96a814f5cae8e60ba10ca9011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:12.701278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.702024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-3e8091c7-7ba2-4088-9120-64cd04c9e656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.703903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1151/2367] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:12 2022] DELETE /identity/v3/projects/ac770ae96a814f5cae8e60ba10ca9011 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:12.709938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.710436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.710775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.742546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:12.743897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.775111 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.776304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.776462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.776738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.777425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.779683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.815561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1217/2368] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:12.816741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.819765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.827835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1152/2369] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:12.827835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:12.829438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.842974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f811c-2fb2-463c-aded-13ceb941e951 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.877930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:12.902817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241016b3-5fa3-4860-8187-3f48ea7ab87e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:12.908498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.909149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.909427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.909813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] PATH_INFO: `/v3/projects/11cf864b785b4b638efbc2eeee879cca` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.910845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=11cf864b785b4b638efbc2eeee879cca)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.914110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.916346 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:12.916979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.917263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.917710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.918350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:12.920352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0c43-773e-4128-9be0-204acaa33dbe admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:12.949270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1153/2370] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/users => generated 288 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:12.949270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72020fb-e0c8-4a6f-bdee-96d97460cf77 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:12.949270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72020fb-e0c8-4a6f-bdee-96d97460cf77 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:12.949654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72020fb-e0c8-4a6f-bdee-96d97460cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:12.981210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a72020fb-e0c8-4a6f-bdee-96d97460cf77 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:12.982518 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a72020fb-e0c8-4a6f-bdee-96d97460cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:12.986135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11cf864b785b4b638efbc2eeee879cca', 'request_id': 'req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:12.998931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11cf864b785b4b638efbc2eeee879cca', 'request_id': 'req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:13.013910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11cf864b785b4b638efbc2eeee879cca', 'request_id': 'req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:13.041945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] The token cache is being invalidate because project 11cf864b785b4b638efbc2eeee879cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:13.042232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:13.042937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:13.044579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1218/2371] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:12 2022] DELETE /identity/v3/projects/11cf864b785b4b638efbc2eeee879cca => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:13.045422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.046778 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:13.064366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:13.119615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1154/2372] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:13.121050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.121408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.121769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.133803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef4f94-4083-4ff9-8596-2cba31a2ab4c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.141123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.141822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.142147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.142461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.143250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.145186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.149841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:13.151227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:13.158342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1219/2373] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.159299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.161601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1155/2374] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:13.163247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.174851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b0a1b-f112-43f9-9d60-426221e8b455 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.178825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597cb7a7-88a4-497f-afe8-5f2e880b3100 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.181044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.181761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.181992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.182276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.182938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.183894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.184514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.184722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f8636-51e4-4118-9eeb-8c7ca36d6ac1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.184861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.185058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.185713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:13.186022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.187647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.201596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1156/2375] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:02:13.201596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.201596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.201596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.203621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1220/2376] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:13.204579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.215841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad293e-caa5-4228-b65f-cf3ae1806e43 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.220649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.220967 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:13.221316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.221594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.221868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] PATH_INFO: `/v3/projects/31af9daac53b462b9db1157b7f288ac6/users/7372b341d7a84c89a077b4e0334a51ec/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.222209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:13.234119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1157/2377] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:13.238459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.242502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31af9daac53b462b9db1157b7f288ac6, user_id=7372b341d7a84c89a077b4e0334a51ec, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.245687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.248665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a589c23-9336-4fd5-8303-712f7cd37ff9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.254358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.255379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.256105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.256604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.257319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.259427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b52912-b826-4ff1-9634-98aae09ba99a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.262199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1221/2378] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:13 2022] PUT /identity/v3/projects/31af9daac53b462b9db1157b7f288ac6/users/7372b341d7a84c89a077b4e0334a51ec/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:13.264662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.303674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1158/2379] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.303674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.303674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.303674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.334035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] MFA Rules not processed for user `7372b341d7a84c89a077b4e0334a51ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:13.335408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:13.351382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60385f-7da8-4a7e-8418-fb688d558f46 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.418161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.418883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.419222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.419685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1159/2380] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:13.419828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.420514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:13.420880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.423009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.424024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.432394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:13.437311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35532ce4-b301-46ae-8e52-7afe4b54052d None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.437907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1222/2381] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:13.438805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.446928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.447552 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.447858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.448135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.448763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.450713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.455883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0a41f-3ad7-4816-a9c8-096cf44d857c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.462424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.463060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.463511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.464075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.465234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.467836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.482588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1160/2382] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/users => generated 293 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.482588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.493559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f46c68-5fc2-46ed-b7cc-f8daa10b6f0f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.493744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:13.499144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.499761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.500070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.500441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.501270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.503776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.514741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1223/2383] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.515900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.549958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1161/2384] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.555530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.599550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f248d1a9-80ca-4a34-9efb-fd0cb6065e68 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.633889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c36aad-ae14-42c1-9ec3-31f7b2b5bc90 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.640156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.640156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.640535 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.640814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.641447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.643375 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.676354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1162/2385] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.677931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.692017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764b9d2-2eda-4b32-9154-ffe0f47b243d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.697144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.697784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.698078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.698341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.699001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.700822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.702558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.703541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.703890 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.704211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.704868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.707271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.715546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1224/2386] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.716499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.728945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdca276-54aa-4f86-8dae-75c40ef9c271 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.734555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.735290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.735638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.736082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.736790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.738812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.750805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1163/2387] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/users => generated 287 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:13.751258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.758225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1225/2388] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.759196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.768224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f648f728-b91f-47a7-83a9-c770b087c22f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.773071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55f666-0b5e-40b5-a87e-4e12ca244b58 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.775625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.776494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.777127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.777555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.778380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.779499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.780052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.780481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.780831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.781172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.781992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.783924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.793969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1164/2389] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.794778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.799356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1226/2390] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.800296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.806540 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e02bcd8-85b8-43e2-8fe6-1460d8822e86 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.811477 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.812272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.812803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.813040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.813326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e4473-1825-4c0c-b8bd-21bba530ab39 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.813641 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.815427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.818816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.819562 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.819941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.820284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] PATH_INFO: `/v3/projects/1a0d20426a9a4c98b68ee87ebfe62623/users/21ac2c8187dc46aaa687e149bd9517b7/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.829755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1165/2391] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.830799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.842151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64fc95-13d8-4c74-9707-3360eaefbe41 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.842380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a0d20426a9a4c98b68ee87ebfe62623, user_id=21ac2c8187dc46aaa687e149bd9517b7, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.845507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a95326-ea8e-4c97-a068-d295b6523388 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.847529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.848367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.849572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.850161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.850613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.852244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.864800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1166/2392] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:13.866266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1227/2393] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:13 2022] PUT /identity/v3/projects/1a0d20426a9a4c98b68ee87ebfe62623/users/21ac2c8187dc46aaa687e149bd9517b7/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:13.868748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.871171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.871436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.871710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.883138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9564c0b-d26c-4a67-885c-49c1191bdd45 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.896962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:13.898441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:13.950296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:13.952656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.954267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.955479 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.955986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] PATH_INFO: `/v3/projects/86ba0f7bc5784f3ba2ec86c6c9cd3645/users/05982b4980f04cec97afbb8a4b1875e2/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:13.967148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:13.975094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1228/2394] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:13.976811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:13.988081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d53041-5543-4360-879a-69bf7e00e258 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:13.988758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86ba0f7bc5784f3ba2ec86c6c9cd3645, user_id=05982b4980f04cec97afbb8a4b1875e2, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:13.992341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:13.993440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:13.994461 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:13.994748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:13.995036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] PATH_INFO: `/v3/projects/7cc6df83bfd54df6b8c34a893e08faa1/users/a08509edf0f14a1c96035684290226f2/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.012694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1167/2395] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:13 2022] PUT /identity/v3/projects/86ba0f7bc5784f3ba2ec86c6c9cd3645/users/05982b4980f04cec97afbb8a4b1875e2/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:14.014244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc6df83bfd54df6b8c34a893e08faa1, user_id=a08509edf0f14a1c96035684290226f2, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.017037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ed03c-a737-4445-a2bc-b5717a815506 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.021765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.023697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:14.032690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1229/2396] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:13 2022] PUT /identity/v3/projects/7cc6df83bfd54df6b8c34a893e08faa1/users/a08509edf0f14a1c96035684290226f2/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:14.034300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f09a-3e47-4b0c-a8f4-6c96330b7f66 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.034589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f09a-3e47-4b0c-a8f4-6c96330b7f66 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.034871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d270f09a-3e47-4b0c-a8f4-6c96330b7f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.069146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d270f09a-3e47-4b0c-a8f4-6c96330b7f66 None None] MFA Rules not processed for user `05982b4980f04cec97afbb8a4b1875e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:14.070446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d270f09a-3e47-4b0c-a8f4-6c96330b7f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:14.099338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f5134-a16b-44ac-8527-aa2e8468083d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.159087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.160054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.160475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.161532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.173524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1230/2397] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:14.174684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.175004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.175157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.204390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:14.205804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:14.263729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.265597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.275998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1168/2398] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:14.276726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.277558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1231/2399] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:14.288045 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.288796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208aea0d-3aab-4f7c-ba6e-c2ce2513c794 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.289664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:14.297686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.299009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.299296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.299572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.300255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.302765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.304100 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2470c67-fb36-43bb-bbd2-797e7780728f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.308379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.309274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.309625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.310013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.320421 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.322277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.334445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1232/2400] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:14.338229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1169/2401] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:14 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:14.339661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.350557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1139759-c4ea-457c-88ea-a0ba12083bfc service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.399982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:14.425803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.427389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.427686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.428651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.429388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.431509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.463019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1233/2402] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:14 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:14.467296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.479693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a269398-8030-44e0-9deb-6c5057b3fe73 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.485113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.485816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.486246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.486588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.487570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.489336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.505408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1170/2403] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:14.509472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.524439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d07b1-5f3d-4b1f-ba5c-8f61469b629c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.531207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.532239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.532628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.533005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] PATH_INFO: `/v3/projects/58279620c08644e39719048dddfc4c04/users/32355913d98347d6aff2e9d0b4accd39/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.563582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58279620c08644e39719048dddfc4c04, user_id=32355913d98347d6aff2e9d0b4accd39, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.569690 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.596399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1234/2404] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:14 2022] PUT /identity/v3/projects/58279620c08644e39719048dddfc4c04/users/32355913d98347d6aff2e9d0b4accd39/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:02:14.604071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.623080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94b628-01fd-4167-8035-a3e1d75f2103 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.647705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:14.702264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.703076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.703434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.703774 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.705213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.708033 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a963006-c478-4b09-b82e-535f4555a41c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.726505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1171/2405] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:14.727579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:14.739891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959953-0a77-47a4-abcb-2806d623ea06 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:14.746024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:14.747088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.747563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.748076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] PATH_INFO: `/v3/projects/58279620c08644e39719048dddfc4c04/users/32355913d98347d6aff2e9d0b4accd39/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.751522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] RBAC: Authorizing `identity:create_grant(project_id=58279620c08644e39719048dddfc4c04, user_id=32355913d98347d6aff2e9d0b4accd39, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:14.754681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:14.770285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1235/2406] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:14 2022] PUT /identity/v3/projects/58279620c08644e39719048dddfc4c04/users/32355913d98347d6aff2e9d0b4accd39/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:14.782678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792527b-99a7-41f3-adaf-3c74d8397be3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:14.783020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792527b-99a7-41f3-adaf-3c74d8397be3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:14.783296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792527b-99a7-41f3-adaf-3c74d8397be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:14.862288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9792527b-99a7-41f3-adaf-3c74d8397be3 None None] MFA Rules not processed for user `32355913d98347d6aff2e9d0b4accd39`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:14.865074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9792527b-99a7-41f3-adaf-3c74d8397be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:15.019116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1172/2407] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:15.032247 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:15.034986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:15.224866 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12610197-753a-4f2b-9797-71b46cc2635f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:15.241894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:15.242552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:15.243021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:15.243358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:15.274199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:15.274541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:15.274817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:15.351765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] MFA Rules not processed for user `a08509edf0f14a1c96035684290226f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:15.353909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:15.415507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:15.417310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:15.446388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1236/2408] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:15.679844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1173/2409] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:15.844865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:15.847612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:15.863476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea205c6-2ec7-4ff7-be53-842d264c7217 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:15.871029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:15.872101 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:15.872711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:15.873312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:15.885692 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:15.892050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c58916-09c3-423d-b952-534c98f7d103 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:15.909511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1237/2410] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:16.483935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:16.485398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:16.591626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:16.659760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b78154f-aacd-422e-b195-2bfa32ab3f40 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:16.665481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:16.666579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:16.666579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:16.666798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] PATH_INFO: `/v3/users/8bce59f5d19a4c468e8fa2e94307b855` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:16.667934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bce59f5d19a4c468e8fa2e94307b855)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:16.669993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:16.741428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bce59f5d19a4c468e8fa2e94307b855'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:16.742170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bce59f5d19a4c468e8fa2e94307b855'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:16.759912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bce59f5d19a4c468e8fa2e94307b855'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:16.769822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bce59f5d19a4c468e8fa2e94307b855'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:16.780198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bce59f5d19a4c468e8fa2e94307b855'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:16.792673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1174/2411] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:16 2022] DELETE /identity/v3/users/8bce59f5d19a4c468e8fa2e94307b855 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:16.795361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:16.795660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:16.796028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:16.816271 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:16.817762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:16.862923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:16.914214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1238/2412] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:16.924037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:16.925546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:17.009005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff176d-fc2e-4a5f-b06b-369f719f686c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:17.013367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:17.014133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:17.014502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:17.014776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:17.028801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:17.030725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:17.044046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1175/2413] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:17.669768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:17.674248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:17.698424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbcc129-21cb-4c5e-ab1f-740f13a386ba None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:17.721281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:17.721967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:17.722208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:17.722480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] PATH_INFO: `/v3/projects/2ce1e37e54a6443c9c0e5518504d3d1a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:17.725863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce1e37e54a6443c9c0e5518504d3d1a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:17.727836 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:17.847646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce1e37e54a6443c9c0e5518504d3d1a', 'request_id': 'req-9a0ec057-be68-418e-a4c9-9e53d41c2a15'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:17.862236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce1e37e54a6443c9c0e5518504d3d1a', 'request_id': 'req-9a0ec057-be68-418e-a4c9-9e53d41c2a15'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:17.873522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce1e37e54a6443c9c0e5518504d3d1a', 'request_id': 'req-9a0ec057-be68-418e-a4c9-9e53d41c2a15'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:17.892119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] The token cache is being invalidate because project 2ce1e37e54a6443c9c0e5518504d3d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:17.892517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:17.893697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:17.895523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1239/2414] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:17 2022] DELETE /identity/v3/projects/2ce1e37e54a6443c9c0e5518504d3d1a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:17.899717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:17.915445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438eb93a-b12b-4d40-a5ae-7d7919cf39ad service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:17.999128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:18.013742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:18.014426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.015099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.015531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] PATH_INFO: `/v3/users/25b4e267a562445c931492f1e3a3108f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.017591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b4e267a562445c931492f1e3a3108f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:18.021133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:18.085281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b4e267a562445c931492f1e3a3108f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.086251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b4e267a562445c931492f1e3a3108f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.102076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b4e267a562445c931492f1e3a3108f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.112067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b4e267a562445c931492f1e3a3108f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.124551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0ae751e8-a8a9-4e7d-a570-efeb42acca24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b4e267a562445c931492f1e3a3108f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.138572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1176/2415] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:17 2022] DELETE /identity/v3/users/25b4e267a562445c931492f1e3a3108f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:18.474565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:18.479409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:18.501837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ec057-be68-418e-a4c9-9e53d41c2a15 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:18.579958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:18.580829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.581618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.582071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] PATH_INFO: `/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.584838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7aa2b00c9e4c97bafd13328a3cdf19)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:18.587670 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:18.669190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7aa2b00c9e4c97bafd13328a3cdf19', 'request_id': 'req-060be0ca-5dca-4b1e-9303-26130355eaa3'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.678398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7aa2b00c9e4c97bafd13328a3cdf19', 'request_id': 'req-060be0ca-5dca-4b1e-9303-26130355eaa3'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.684981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7aa2b00c9e4c97bafd13328a3cdf19', 'request_id': 'req-060be0ca-5dca-4b1e-9303-26130355eaa3'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.686407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:18.701194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] The token cache is being invalidate because project ef7aa2b00c9e4c97bafd13328a3cdf19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:18.701626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.702837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-060be0ca-5dca-4b1e-9303-26130355eaa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:18.706181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1240/2416] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:18 2022] DELETE /identity/v3/projects/ef7aa2b00c9e4c97bafd13328a3cdf19 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:18.711742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.715146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.715146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.752795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:18.754039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:18.855203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1177/2417] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:18.863057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.863613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.864174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.887975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:18.890011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:18.902221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1241/2418] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:18.906787 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:18.919561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd065b-012d-472f-a745-9dee14401536 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:18.925896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:18.926721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.927023 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.927253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.927937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:18.928242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:18.929923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:18.946721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1178/2419] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:02:18.948513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:18.960947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c283c0-402c-459a-bd96-9603c4d2de63 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:18.966170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:18.966802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:18.967086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:18.967356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:18.968021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:18.969949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.012641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1242/2420] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:18 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:19.013771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.029000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4a2fe-12e3-4599-922d-01cb3e51b731 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.099185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.099924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.100279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.100613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.101314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.103351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.111927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:19.133839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1179/2421] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:19 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:19.138030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.149258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6ec62-6302-420c-8add-0af28932d8a5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.154033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.154642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.154927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.155203 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.156012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.157637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.168446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1243/2422] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:19.173148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.190175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259f222-1381-4ff6-951e-c4101651aa6a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.200734 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.202144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.202144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.202144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.203440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.205378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eaae35-6ea4-4648-99e6-db4b18efbf94 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.227601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1180/2423] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:19.228807 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.240668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1433f-f5e4-42b2-b947-663fef286d7b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.245796 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.246482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.246763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.247040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] PATH_INFO: `/v3/projects/b0ade0848b784b4e8eaf4e136fd0893d/users/918ec69f11a44d7d88245efbbc632d03/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.265210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ade0848b784b4e8eaf4e136fd0893d, user_id=918ec69f11a44d7d88245efbbc632d03, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.267969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.284114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1244/2424] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:19 2022] PUT /identity/v3/projects/b0ade0848b784b4e8eaf4e136fd0893d/users/918ec69f11a44d7d88245efbbc632d03/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:19.295780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.296096 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.296366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.329258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] MFA Rules not processed for user `918ec69f11a44d7d88245efbbc632d03`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:19.330541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:19.424826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1181/2425] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:19.431357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.433635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:19.518438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da12296-f321-46f1-8101-ea9cca3efad6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.522718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.523342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.523627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.523971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.533028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.534476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31641950-2103-4937-9303-6101b73dbc52 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.544856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1245/2426] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:19.565367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:19.681106 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7608e8-753e-4b4c-b448-24a2975c5f51 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:19.689866 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:19.690216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.690494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.690760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] PATH_INFO: `/v3/users/918ec69f11a44d7d88245efbbc632d03` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.694171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] RBAC: Authorizing `identity:delete_user(user_id=918ec69f11a44d7d88245efbbc632d03)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:19.697674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:19.818767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918ec69f11a44d7d88245efbbc632d03'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:19.820628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918ec69f11a44d7d88245efbbc632d03'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:19.840355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918ec69f11a44d7d88245efbbc632d03'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:19.844977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:19.856414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918ec69f11a44d7d88245efbbc632d03'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:19.872405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918ec69f11a44d7d88245efbbc632d03'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:19.884047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1182/2427] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:19 2022] DELETE /identity/v3/users/918ec69f11a44d7d88245efbbc632d03 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:19.888473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:19.889491 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:19.889987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:19.921360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:19.924118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:20.023899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1246/2428] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:20.061066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:20.063258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:20.272111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c23d15-adf0-4332-ba03-362e84caf83b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:20.276577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:20.277235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:20.277514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:20.277786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:20.300236 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:20.302864 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343c252-3339-4eb2-8e36-33eb5d45331f service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:20.329510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1183/2429] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:21.538704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:21.540380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:21.558289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef67cf-b9f7-49ed-b38b-60129c2acab6 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:21.572629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:21.573522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:21.576954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:21.577521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] PATH_INFO: `/v3/projects/b0ade0848b784b4e8eaf4e136fd0893d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:21.580568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ade0848b784b4e8eaf4e136fd0893d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:21.583863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:21.661687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:21.742254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ade0848b784b4e8eaf4e136fd0893d', 'request_id': 'req-8d92041c-5053-42b3-bfe4-bf9d52168860'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:21.774672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ade0848b784b4e8eaf4e136fd0893d', 'request_id': 'req-8d92041c-5053-42b3-bfe4-bf9d52168860'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:21.783132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ade0848b784b4e8eaf4e136fd0893d', 'request_id': 'req-8d92041c-5053-42b3-bfe4-bf9d52168860'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:21.817673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] The token cache is being invalidate because project b0ade0848b784b4e8eaf4e136fd0893d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:21.817982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:21.819696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8d92041c-5053-42b3-bfe4-bf9d52168860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:21.839328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1247/2430] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:21 2022] DELETE /identity/v3/projects/b0ade0848b784b4e8eaf4e136fd0893d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:21.844282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:21.844845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:21.844845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:21.896084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:21.897923 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.043886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1184/2431] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:22.057871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.058305 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.059057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.089610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:22.092189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.094216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.104283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.131392 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1248/2432] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:22.135935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.161942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479a830-204d-4330-ac74-60af45a4efe9 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.174722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.176251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.178024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.179038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.180783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:22.181529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.185849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.220074 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1249/2433] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:22.224240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.247942 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd852-7aa6-4cdc-9999-c4e2a8950780 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.250925 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fd396-f689-4303-bbfa-4ae7ea52ebbb admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.268699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.270117 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.275193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.275337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.276112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.276470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.276803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.277292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.278348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.279782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.280754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.337370 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1250/2434] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/projects => generated 391 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:22.341974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.416355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:22.451039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b5f3d-d7fa-442e-b6b1-43b6185a07f4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.455497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.456193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.456478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.456754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.457680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.469890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.477381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.478860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f5381d-c8d4-4fbd-a09d-e4f73c507118 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.499732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1185/2435] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:02:22.514897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.515178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.515439 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.533136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1251/2436] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:22.538748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.555175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203aac4-0827-4741-bb89-77437bee732c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.565225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.567578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.568106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.568587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.569451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.573176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.596332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] MFA Rules not processed for user `21ac2c8187dc46aaa687e149bd9517b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:22.596696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1252/2437] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:22.597610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.600155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.637021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b0408-6fd8-4274-976a-de11fe312bc2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.651265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.652449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.652728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.653004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.653752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.659081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.703672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1253/2438] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:22.708914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.734456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d61ce7-855b-4c41-a76d-3261ad0f8959 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.755885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:22.756573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.757033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.757229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] PATH_INFO: `/v3/projects/b15502eec6fd4ddb95db8afdf94bcbfb/users/42828446501348d3bbf109926879bf4b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.798761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15502eec6fd4ddb95db8afdf94bcbfb, user_id=42828446501348d3bbf109926879bf4b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:22.802098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64bdda6-bef3-49fc-892f-1262e6502b9d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:22.810393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1186/2439] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:22.816230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:22.821686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:22.834714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1254/2440] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:22 2022] PUT /identity/v3/projects/b15502eec6fd4ddb95db8afdf94bcbfb/users/42828446501348d3bbf109926879bf4b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:22.846174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b763c-363d-4562-804d-ab2764303106 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:22.846739 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b763c-363d-4562-804d-ab2764303106 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:22.847174 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0b763c-363d-4562-804d-ab2764303106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:22.872538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26de8899-a64b-49fc-8031-92a036668e52 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:22.898055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:22.905362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-9c0b763c-363d-4562-804d-ab2764303106 None None] MFA Rules not processed for user `42828446501348d3bbf109926879bf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:22.909525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c0b763c-363d-4562-804d-ab2764303106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:23.027571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.030260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.031500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.032597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.033543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.037502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.052593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1255/2441] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:23.083837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.084220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.084615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.110440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1187/2442] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:22 2022] POST /identity/v3/projects => generated 385 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:23.115689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.133119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] MFA Rules not processed for user `a08509edf0f14a1c96035684290226f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:23.137593 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:23.238985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1256/2443] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:23.238985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.240869 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:23.255005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ea614-1ff7-4f97-947f-62ba793231a7 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.266799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.267412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.267756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.268269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.269922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.272297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.282185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:23.317857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1188/2444] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:23 2022] POST /identity/v3/users => generated 299 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:23.319696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.337463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58151500-bb2c-45bb-9112-9598fa64e025 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.348752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.349737 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.350192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.350651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.351551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.353610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.385979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1189/2445] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:23.389806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.408588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4ce60-3314-4003-939d-bc4076226dc4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.414079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.414710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.415241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.415520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.418046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640abf7-00f7-44b3-ae7f-5290e6c5b8ac admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.429065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.429770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.430215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.430487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] PATH_INFO: `/v3/projects/07a316770d43428883ed16509c4c3cb5/users/5a483c3f4c4b4968bcd2d575b5929246/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.468656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a316770d43428883ed16509c4c3cb5, user_id=5a483c3f4c4b4968bcd2d575b5929246, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.473536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.500754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1190/2446] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:23 2022] PUT /identity/v3/projects/07a316770d43428883ed16509c4c3cb5/users/5a483c3f4c4b4968bcd2d575b5929246/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:23.502000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.540343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.541834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.562746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1257/2447] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:23.645591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10486a-3390-4a2e-abaa-0751c2fd5c4b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.676700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.680735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.681100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.681458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.682272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.687197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.719443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1191/2448] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:23.719667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:23.725612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:23.735438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708ff44-67d1-4e4b-8988-78c4d842ee3e service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:23.741941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:23.742726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:23.743885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:23.744656 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] PATH_INFO: `/v3/projects/07a316770d43428883ed16509c4c3cb5/users/5a483c3f4c4b4968bcd2d575b5929246/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:23.746343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07a316770d43428883ed16509c4c3cb5, user_id=5a483c3f4c4b4968bcd2d575b5929246, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:23.749804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c079fd-5083-499f-81a4-544633ccfd69 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:23.768471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1258/2449] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:23 2022] PUT /identity/v3/projects/07a316770d43428883ed16509c4c3cb5/users/5a483c3f4c4b4968bcd2d575b5929246/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:02:24.107993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:24.183679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfe3f8-9285-4533-8bf3-f75eee7bb311 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:24.249653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:24.250430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:24.250815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:24.251179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:24.254563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:24.293430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:24.434474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:24.435867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:24.468574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1192/2450] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:29.672347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa355f-3476-4804-bf1a-d8c25708851d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:29.672772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa355f-3476-4804-bf1a-d8c25708851d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:29.672890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa355f-3476-4804-bf1a-d8c25708851d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:29.707320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fcaa355f-3476-4804-bf1a-d8c25708851d None None] MFA Rules not processed for user `5a483c3f4c4b4968bcd2d575b5929246`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:29.708615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fcaa355f-3476-4804-bf1a-d8c25708851d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:29.794826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1259/2451] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:02:29.807919 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:29.809459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:29.933608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c4398d-d5f1-432b-8a3c-525bebbdd256 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:29.938321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:29.938948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:29.939227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:29.939500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:30.028914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:30.030397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:30.041315 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1193/2452] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:30.172664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1e5f-12f7-4a4b-add1-6559754e5ae6 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:30.173062 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1e5f-12f7-4a4b-add1-6559754e5ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:30.173403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e1e5f-12f7-4a4b-add1-6559754e5ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:30.195556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f03e1e5f-12f7-4a4b-add1-6559754e5ae6 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:30.196930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f03e1e5f-12f7-4a4b-add1-6559754e5ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:30.291721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1260/2453] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:30.306522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:30.308103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:30.435110 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f921010-a06e-4074-9785-9f21cab884a1 service glance] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:30.439637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:30.442330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:30.442849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:30.443127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:30.461222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:30.464001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:30.469974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:30.477871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1194/2454] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:02:40.131953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96788a-76be-4452-a884-3b46e9471597 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:40.132523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96788a-76be-4452-a884-3b46e9471597 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:40.132636 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96788a-76be-4452-a884-3b46e9471597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:40.157039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2c96788a-76be-4452-a884-3b46e9471597 None None] MFA Rules not processed for user `21ac2c8187dc46aaa687e149bd9517b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:40.159314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2c96788a-76be-4452-a884-3b46e9471597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:40.168292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1261/2455] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:50.149632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=66,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:51.445715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:51.449091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:51.572769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1e326-ea70-4a7c-b113-cb974ada6116 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:51.578759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:51.579320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:51.579670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:51.580049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] PATH_INFO: `/v3/users/7372b341d7a84c89a077b4e0334a51ec` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:51.582247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] RBAC: Authorizing `identity:delete_user(user_id=7372b341d7a84c89a077b4e0334a51ec)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:51.584464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:51.703429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7372b341d7a84c89a077b4e0334a51ec'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:51.707724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7372b341d7a84c89a077b4e0334a51ec'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:51.750637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdab8-735d-463b-a94e-67bc3f80ef3c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:51.751149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdab8-735d-463b-a94e-67bc3f80ef3c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:51.751326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcdab8-735d-463b-a94e-67bc3f80ef3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:51.761051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7372b341d7a84c89a077b4e0334a51ec'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:51.788859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7372b341d7a84c89a077b4e0334a51ec'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:51.792765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5bdcdab8-735d-463b-a94e-67bc3f80ef3c None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:51.795173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5bdcdab8-735d-463b-a94e-67bc3f80ef3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:51.804968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1262/2456] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:02:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:51.808871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7372b341d7a84c89a077b4e0334a51ec'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:51.817976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1195/2457] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:51 2022] DELETE /identity/v3/users/7372b341d7a84c89a077b4e0334a51ec => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:51.822462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:51.822757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:51.823028 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:51.825332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:51.827647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:51.848992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:51.850642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:51.927199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bcddc-428e-4fd9-a347-39f319b9c9be admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:51.943295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:51.944495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:51.945568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:51.946005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:51.950540 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1263/2458] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:51.973379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:51.976979 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:51.979829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:52.032752 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:52.035150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:52.048476 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1196/2459] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:02:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:02:52.061765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48a1c90-6ca0-4e84-96b3-f2ab6c33f4cc None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:52.068673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:52.068673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:52.068903 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:52.069353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:52.086627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:52.086627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:52.112208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1264/2460] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:52.985766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:52.987639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.012888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677b8c3-909a-4d6b-9b2b-8bc32daaeb43 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.033001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.034268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.035229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.035814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] PATH_INFO: `/v3/projects/31af9daac53b462b9db1157b7f288ac6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.044708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31af9daac53b462b9db1157b7f288ac6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.046694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.199773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31af9daac53b462b9db1157b7f288ac6', 'request_id': 'req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.220448 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31af9daac53b462b9db1157b7f288ac6', 'request_id': 'req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.251215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31af9daac53b462b9db1157b7f288ac6', 'request_id': 'req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.304012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] The token cache is being invalidate because project 31af9daac53b462b9db1157b7f288ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:53.305041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.306690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.314017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1197/2461] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:52 2022] DELETE /identity/v3/projects/31af9daac53b462b9db1157b7f288ac6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:53.318985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.339887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec1066-67dc-4f3d-aacd-bc71092c21a6 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:53.467091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:53.468663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:53.468946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:53.469390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] PATH_INFO: `/v3/users/05982b4980f04cec97afbb8a4b1875e2` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:53.474570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] RBAC: Authorizing `identity:delete_user(user_id=05982b4980f04cec97afbb8a4b1875e2)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:53.478695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:53.571225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:53.608733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05982b4980f04cec97afbb8a4b1875e2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.610448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05982b4980f04cec97afbb8a4b1875e2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.625390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05982b4980f04cec97afbb8a4b1875e2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.651826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05982b4980f04cec97afbb8a4b1875e2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.670881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-dc0c441c-b361-41a0-af15-982f24d9036f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05982b4980f04cec97afbb8a4b1875e2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:53.706741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1265/2462] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:53 2022] DELETE /identity/v3/users/05982b4980f04cec97afbb8a4b1875e2 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:53.853563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:53.855133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:53.884449 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ed1d5-2322-4fd2-9cb3-c766c33b6e62 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.054605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.055472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.055622 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.055876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] PATH_INFO: `/v3/users/613c09ab1ac94b17bd81e855c09a51ea` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.056979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=613c09ab1ac94b17bd81e855c09a51ea)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.065397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.187901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.191085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '613c09ab1ac94b17bd81e855c09a51ea'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.198157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '613c09ab1ac94b17bd81e855c09a51ea'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.248212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '613c09ab1ac94b17bd81e855c09a51ea'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.267495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '613c09ab1ac94b17bd81e855c09a51ea'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.277251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '613c09ab1ac94b17bd81e855c09a51ea'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.285353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1198/2463] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:53 2022] DELETE /identity/v3/users/613c09ab1ac94b17bd81e855c09a51ea => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:54.290496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.291927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.292048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.292188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.292921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.324184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:54.325539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.378871 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd1fc3c-3973-49b5-b02f-6be5ff5d0941 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.384768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.385408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.385690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.385971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] PATH_INFO: `/v3/projects/86ba0f7bc5784f3ba2ec86c6c9cd3645` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.387125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=86ba0f7bc5784f3ba2ec86c6c9cd3645)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.389121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.397665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1266/2464] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:54.409075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.411131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.477168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86ba0f7bc5784f3ba2ec86c6c9cd3645', 'request_id': 'req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.491339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86ba0f7bc5784f3ba2ec86c6c9cd3645', 'request_id': 'req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.506685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834015b1-4cd6-47ff-8987-19be70fdf373 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.510319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86ba0f7bc5784f3ba2ec86c6c9cd3645', 'request_id': 'req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.528253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] The token cache is being invalidate because project 86ba0f7bc5784f3ba2ec86c6c9cd3645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:54.528798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.529731 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.532259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1199/2465] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:54 2022] DELETE /identity/v3/projects/86ba0f7bc5784f3ba2ec86c6c9cd3645 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:54.538289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:54.539756 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.572592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.573479 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.573891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.574396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.576866 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:54.602615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.622326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7500e-ae6f-48d1-91b9-8a7d738b6b9e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:54.632188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:54.632990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:54.634618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:54.635198 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:54.636426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] PATH_INFO: `/v3/users/32355913d98347d6aff2e9d0b4accd39` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:54.639076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] RBAC: Authorizing `identity:delete_user(user_id=32355913d98347d6aff2e9d0b4accd39)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.645099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.700429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:54.702175 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:54.718334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1267/2466] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:54.735286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32355913d98347d6aff2e9d0b4accd39'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.736069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32355913d98347d6aff2e9d0b4accd39'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.776276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32355913d98347d6aff2e9d0b4accd39'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.789289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32355913d98347d6aff2e9d0b4accd39'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.801671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32355913d98347d6aff2e9d0b4accd39'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:54.814780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1200/2467] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:54 2022] DELETE /identity/v3/users/32355913d98347d6aff2e9d0b4accd39 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.392239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.394787 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.433583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aedd769-98dd-43a0-8de7-b67c7c4696dc service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.508366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.510175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.617970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.618614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.619083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.619523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] PATH_INFO: `/v3/projects/2375176960a245cea212397a4a5babb0` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.621717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2375176960a245cea212397a4a5babb0)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.624193 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.741443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d17953-9491-41d0-af93-6ae875d8f05d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.755106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2375176960a245cea212397a4a5babb0', 'request_id': 'req-96066e86-0051-41c1-aa17-78a34177bb52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.772788 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2375176960a245cea212397a4a5babb0', 'request_id': 'req-96066e86-0051-41c1-aa17-78a34177bb52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.782909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2375176960a245cea212397a4a5babb0', 'request_id': 'req-96066e86-0051-41c1-aa17-78a34177bb52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.802669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] The token cache is being invalidate because project 2375176960a245cea212397a4a5babb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:55.803231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.804354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-96066e86-0051-41c1-aa17-78a34177bb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.809947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1268/2468] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:55 2022] DELETE /identity/v3/projects/2375176960a245cea212397a4a5babb0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:55.811702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86cc4-a752-4360-913e-c2b35eaf08fa None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.812156 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86cc4-a752-4360-913e-c2b35eaf08fa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.812489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef86cc4-a752-4360-913e-c2b35eaf08fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.853345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8ef86cc4-a752-4360-913e-c2b35eaf08fa None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.856047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8ef86cc4-a752-4360-913e-c2b35eaf08fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.876436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:55.881637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.882788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.883200 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] PATH_INFO: `/v3/projects/58279620c08644e39719048dddfc4c04` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.885478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=58279620c08644e39719048dddfc4c04)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:55.889112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:55.889529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1269/2469] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.904587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:55.904944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:55.906112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:55.931727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:55.947288 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:55.948681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:55.967985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1270/2470] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:55.971590 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:55.982321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58279620c08644e39719048dddfc4c04', 'request_id': 'req-daa1170a-83b1-488f-85ea-4f2f18a509ab'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:55.993025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33369eb0-8df6-4eae-b3ca-996ac56dda7c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:55.996819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58279620c08644e39719048dddfc4c04', 'request_id': 'req-daa1170a-83b1-488f-85ea-4f2f18a509ab'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.013499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58279620c08644e39719048dddfc4c04', 'request_id': 'req-daa1170a-83b1-488f-85ea-4f2f18a509ab'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.040601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] The token cache is being invalidate because project 58279620c08644e39719048dddfc4c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:56.041428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.043203 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-daa1170a-83b1-488f-85ea-4f2f18a509ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.046761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1201/2471] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:55 2022] DELETE /identity/v3/projects/58279620c08644e39719048dddfc4c04 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:56.054677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466898c4-dc54-4fdf-9c92-c75d5ffbcb8e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.055392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466898c4-dc54-4fdf-9c92-c75d5ffbcb8e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.056048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466898c4-dc54-4fdf-9c92-c75d5ffbcb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.072801 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.087897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-466898c4-dc54-4fdf-9c92-c75d5ffbcb8e None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.089438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-466898c4-dc54-4fdf-9c92-c75d5ffbcb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.101503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.102138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.102563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.102847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.103511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:56.104129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.105488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.123838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1271/2472] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:56.127757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.134249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.157109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771687f3-291b-41ae-9487-0b7ded67b2ae admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.172299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.174560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.176380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.178996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.181031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.186440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.217048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1202/2473] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.237709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.238663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.239401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.259710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1272/2474] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:56.264120 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.276339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.278070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.350357 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04fa092-563c-4b5a-967e-d19e0afe5626 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.359945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1203/2475] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:56.364350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.365021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.365306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.365580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.366218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.368329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.368884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.398018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1273/2476] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:56.401430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9165f-c71d-44ca-9b2f-6e1bb53bc78f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.402439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.413851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.414660 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.415209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.415971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.416851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:56.417667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.422053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.428424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cdd44-e34e-4e2e-821a-12fc8f58f8f0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.442743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.445245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.445741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.446179 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.447031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.448867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.462748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1204/2477] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:56.468738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.482849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1274/2478] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:56.484546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.490897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773c650-f4e8-4c24-91c9-2060299e1eba admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.498470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49c006-505d-459d-8863-25d9c5ca635b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.498629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.499982 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.500401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.500762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.501588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.505347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.506046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.506751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.507125 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.507458 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.508240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.509935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.523478 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1275/2479] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:56.538587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.562355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecda1a2-7245-4134-b3b2-b5a93cc73138 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.578548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1205/2480] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:56.583702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.584127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.585129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.585613 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.586085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] PATH_INFO: `/v3/projects/788e7b2629ae41f7bd9e64f32744c8d8/users/0a58a04dd69a4078a820081847c4e99f/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.618746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.635220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=788e7b2629ae41f7bd9e64f32744c8d8, user_id=0a58a04dd69a4078a820081847c4e99f, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.638566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.665505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1276/2481] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:56 2022] PUT /identity/v3/projects/788e7b2629ae41f7bd9e64f32744c8d8/users/0a58a04dd69a4078a820081847c4e99f/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:56.677825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:56.685571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.743814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258503c-248f-4d3b-9299-009a026cd0cd admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.835817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:56.849003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea33d3d-0def-46eb-b299-1be0bdc592b6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:56.868413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.868651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:56.869264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.869677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.869825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.870270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.870393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.870927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.871183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] PATH_INFO: `/v3/users/42828446501348d3bbf109926879bf4b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.873262 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] RBAC: Authorizing `identity:delete_user(user_id=42828446501348d3bbf109926879bf4b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:56.873551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864fa749-61e2-4bb4-b003-fd8558503554 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.875222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:56.902786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1206/2482] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/users => generated 287 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:02:56.902786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de6d3-ac1f-4633-9530-5b8d41dc4c92 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:56.902786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de6d3-ac1f-4633-9530-5b8d41dc4c92 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:56.902786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379de6d3-ac1f-4633-9530-5b8d41dc4c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:56.938669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-379de6d3-ac1f-4633-9530-5b8d41dc4c92 None None] MFA Rules not processed for user `0a58a04dd69a4078a820081847c4e99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:56.940769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-379de6d3-ac1f-4633-9530-5b8d41dc4c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:56.955779 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42828446501348d3bbf109926879bf4b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.957123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42828446501348d3bbf109926879bf4b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.971164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42828446501348d3bbf109926879bf4b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:56.986662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42828446501348d3bbf109926879bf4b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:57.014735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42828446501348d3bbf109926879bf4b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:57.028955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1277/2483] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:02:56 2022] DELETE /identity/v3/users/42828446501348d3bbf109926879bf4b => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:57.028955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.028955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.091808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1207/2484] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:57.091808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.091808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.091808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.130342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.132087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.195263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.196415 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5870e4-3ce4-43d5-8cf1-4dfa73fbe50d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.202342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.203172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.203555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.203923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.204754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.207289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.223512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1278/2485] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:57.226212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.234887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.244875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1208/2486] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:57.245921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.264259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad2650-f236-4c84-bcfb-5ec75de017ba None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.269617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.270446 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.270741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.271007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.271677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.274576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.286578 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1209/2487] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:57.287646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.291046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.335131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cbcc7-54a8-4e71-992c-f7fdd9f95ed4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.338731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.340500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.340500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.340995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.365429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd21b-188e-4266-90e7-567a0c764df5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.372649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.373382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.373668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.374453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.387379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.388941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.403302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1210/2488] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.404629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.419069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38367c-e2f9-4ed8-953a-b044b88f5c36 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.419303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:57.432511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.433879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.434538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.435101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] PATH_INFO: `/v3/projects/a7a284ee619946b28f3e70a1aa8d2662/users/5f392617d7db4aba8511603ca19ae462/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.475063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a284ee619946b28f3e70a1aa8d2662, user_id=5f392617d7db4aba8511603ca19ae462, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.476130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.478071 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a08762-ccf2-48c8-9bf0-23b94b940a22 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.479928 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.493551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1279/2489] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:57.509339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1211/2490] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:57 2022] PUT /identity/v3/projects/a7a284ee619946b28f3e70a1aa8d2662/users/5f392617d7db4aba8511603ca19ae462/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:57.511306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.511691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.512115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.551092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:57.553444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:57.767401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1280/2491] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:57.799520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:57.920487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b94a4-8b06-4a7a-a097-860be5b6fb6e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:57.935722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:57.937004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:57.937364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:57.937710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:57.953227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:57.956891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13f783-0ad1-4bf2-851d-41593d911088 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:57.972888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1212/2492] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:02:58.404855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:58.412969 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:58.429295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733b9c-cc28-46e8-aa70-72cce9f40771 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:58.434589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:58.435299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.435652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.435893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] PATH_INFO: `/v3/projects/b15502eec6fd4ddb95db8afdf94bcbfb` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:58.437009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15502eec6fd4ddb95db8afdf94bcbfb)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:58.439710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:58.565501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15502eec6fd4ddb95db8afdf94bcbfb', 'request_id': 'req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:58.592468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15502eec6fd4ddb95db8afdf94bcbfb', 'request_id': 'req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:58.608595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15502eec6fd4ddb95db8afdf94bcbfb', 'request_id': 'req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:58.625337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] The token cache is being invalidate because project b15502eec6fd4ddb95db8afdf94bcbfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:02:58.625644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:58.627826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-8ebfcd0f-7f2b-4fa3-9504-2f0c50169007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:02:58.631022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1281/2493] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:02:58 2022] DELETE /identity/v3/projects/b15502eec6fd4ddb95db8afdf94bcbfb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:02:58.637481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.638054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.639233 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:58.680297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:58.681847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:58.875642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:58.938337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1213/2494] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:58.940462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:58.940957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:58.942063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.012721 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:59.014059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:02:59.047023 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1282/2495] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:02:59.054004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.068584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a49b91-84fd-4470-bf2c-e52f56d0b3ab None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.103069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.103069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.103668 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.104108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.104822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:02:59.105816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.111671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.163704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1214/2496] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:02:59.165817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.198076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bcbb9-ecf4-4c8f-bb82-35fae1169470 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.210688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.211880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.212350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.212822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.213989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.218160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.306740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1283/2497] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:02:59 2022] POST /identity/v3/projects => generated 399 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:59.308064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.324682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41eefe-3cd0-4a5c-9df4-991c3a95dc24 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.402074 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.403137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.403681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.404208 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.405238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.410184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.478459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1215/2498] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:02:59 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:02:59.482130 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.494232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acf439-6973-4520-9d03-ce0323677a31 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.529164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.529800 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.530080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.530353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.531080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.532769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.569316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1284/2499] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:02:59.574132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.603109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747510f5-a294-482d-ad9e-1f0f301f52cc admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.622879 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.625207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.627648 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.628136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.629004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.634874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c11381-6237-48f9-a0ee-0280217ea13b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.678613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1216/2500] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:02:59.681221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:02:59.697612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1149a-71de-4f35-9d81-c3885ec11ae5 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:02:59.715232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:02:59.716293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.716808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.717258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] PATH_INFO: `/v3/projects/32d23b54bac549a1843099b477ed8cd3/users/351563957ef744bdb482bc9c303e05f2/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.723293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:02:59.761018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32d23b54bac549a1843099b477ed8cd3, user_id=351563957ef744bdb482bc9c303e05f2, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:02:59.776462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:02:59.845163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1285/2501] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:02:59 2022] PUT /identity/v3/projects/32d23b54bac549a1843099b477ed8cd3/users/351563957ef744bdb482bc9c303e05f2/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:02:59.858282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:02:59.859427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:02:59.859977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:02:59.949051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] MFA Rules not processed for user `351563957ef744bdb482bc9c303e05f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:02:59.950421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:00.087197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1217/2502] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:02:59 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:00.195536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:00.197404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:00.373857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca2a6-5dcb-4ad6-981b-e21d67d21ec2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:00.399024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:00.399024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:00.401359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:00.401679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:00.435020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:00.453838 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad72eec-991d-4279-a822-d5cda3e16180 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:00.535530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1286/2503] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:00 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:03.457123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:03.459225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:03.588472 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:03.614263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c067c6-bd65-4b6e-8f29-a066bafb4b47 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:03.623536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:03.625856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:03.627043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:03.629084 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] PATH_INFO: `/v3/users/0a58a04dd69a4078a820081847c4e99f` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:03.631941 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a58a04dd69a4078a820081847c4e99f)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:03.634860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:03.803132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a58a04dd69a4078a820081847c4e99f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.804138 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a58a04dd69a4078a820081847c4e99f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.826247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a58a04dd69a4078a820081847c4e99f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.842027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a58a04dd69a4078a820081847c4e99f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.852649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a58a04dd69a4078a820081847c4e99f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:03.864331 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1218/2504] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:03 2022] DELETE /identity/v3/users/0a58a04dd69a4078a820081847c4e99f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:03.870033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:03.870954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:03.871524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:03.911419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:03.917260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:04.099336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1287/2505] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:04.136160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:04.137677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:04.295860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902124a-3ed0-434c-9e8a-d5e1eee8f11a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:04.306659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:04.307327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:04.307747 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:04.307987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:04.337214 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:04.343161 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab93c2-b8b5-4935-ab4c-3f5806e6368f service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:04.368583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1219/2506] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:05.102228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.105401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:05.125788 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd04c4-0782-4f73-82a1-d73115921d37 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.132522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.133176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.133457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.133907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] PATH_INFO: `/v3/projects/788e7b2629ae41f7bd9e64f32744c8d8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.135106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=788e7b2629ae41f7bd9e64f32744c8d8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.137155 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.200809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788e7b2629ae41f7bd9e64f32744c8d8', 'request_id': 'req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.210789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788e7b2629ae41f7bd9e64f32744c8d8', 'request_id': 'req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.219326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788e7b2629ae41f7bd9e64f32744c8d8', 'request_id': 'req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.236103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] The token cache is being invalidate because project 788e7b2629ae41f7bd9e64f32744c8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:05.236667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.237424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1834ef1-9375-42eb-9f02-d9b03bddc3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:05.239888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1288/2507] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:05 2022] DELETE /identity/v3/projects/788e7b2629ae41f7bd9e64f32744c8d8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:05.244577 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.245575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.245906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.272450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:05.273700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:05.350012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1220/2508] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:05.351797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.352293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.352417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.384076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:05.386034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:05.400422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1289/2509] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:05.402801 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.419310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae424b9-924c-47f5-8586-22b4d8a2f61f None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.430113 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.431083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.431566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.433049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.434076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:05.435058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.437497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.451245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:05.467251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1221/2510] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:05.472694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.487926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:05.495011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769766b-3ac7-40a0-83e8-eed780e16dca None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.503887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.504531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.504827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.505722 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.506493 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.510128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.557085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1290/2511] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:05.558563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.570972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee6c36-8c77-439f-88ae-b286325d2ff4 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.664978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.665603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.666426 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.666711 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.667356 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.670773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.718876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1222/2512] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:05.723339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.737842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8cab2-29a4-48e8-8988-c1931fac20dc admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.748963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.749675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.750017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.750360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.751756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.754015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.768067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1291/2513] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:05.771917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.783239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b341f5b0-590c-4602-a170-410cb64210be admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.790379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.791617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.792069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.792261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.792972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.802584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37481f9-cdcd-4c50-aed1-2ce8ede551c7 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.831266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1223/2514] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:05.832994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:05.850093 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d6f9c2-8f00-4c8e-9d67-a546281d9634 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:05.860716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:05.862214 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.863187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.863761 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] PATH_INFO: `/v3/projects/a529db680ed84bd081e73459a45e388d/users/f881c4404dba4b08ab64ed1f501542d4/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.886501 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a529db680ed84bd081e73459a45e388d, user_id=f881c4404dba4b08ab64ed1f501542d4, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:05.890645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:05.916680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1292/2515] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:05 2022] PUT /identity/v3/projects/a529db680ed84bd081e73459a45e388d/users/f881c4404dba4b08ab64ed1f501542d4/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:05.922757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953535a-10b7-4b2f-bd66-7cbbc409b513 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:05.924207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953535a-10b7-4b2f-bd66-7cbbc409b513 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:05.924639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953535a-10b7-4b2f-bd66-7cbbc409b513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:05.969317 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-1953535a-10b7-4b2f-bd66-7cbbc409b513 None None] MFA Rules not processed for user `f881c4404dba4b08ab64ed1f501542d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:05.971858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1953535a-10b7-4b2f-bd66-7cbbc409b513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.074553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1224/2516] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:06.089916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:06.094060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:06.220826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40139d1a-1d5c-4d3b-8e00-6c138fcb19f1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:06.225710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:06.226345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:06.226625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:06.226893 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:06.238822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:06.241166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:06.266360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1293/2517] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:07.469047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.469507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.470300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.500345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] MFA Rules not processed for user `5f392617d7db4aba8511603ca19ae462`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:07.502679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:07.564276 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:07.574287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1225/2518] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:03:07.578719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.580794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:07.646582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ad26a0-75e7-4e9c-8aa5-a279e6f656d9 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.651161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.651773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.652092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.652368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.653011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.654947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.684599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1294/2519] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:07 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:07.689397 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.700718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a4d73-9127-4649-bf8c-23629e695122 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.767210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.768000 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.768425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.768707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.769344 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.771327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.821534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1226/2520] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:07 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:07.825907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.840560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4254fa0-30ba-4738-869a-1d441a11078c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.847010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.848220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.848407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.848611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.849326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.851486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.869849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.870494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1295/2521] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:03:07.888345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de637efe-a557-429b-bb6f-00a1ad131de8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:07.897592 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:07.898434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:07.898798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:07.899101 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] PATH_INFO: `/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e/users/d6ec50a600a04728bc683146aa1877ec/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:07.918445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ac6e6134fd4cf49eb5ac830dbfa21e, user_id=d6ec50a600a04728bc683146aa1877ec, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:07.921393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:07.945261 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1227/2522] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:07 2022] PUT /identity/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e/users/d6ec50a600a04728bc683146aa1877ec/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:07.950952 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:07.967154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f33725-eaea-49c7-a3b5-1f260368c9a1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:08.004209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:08.084452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:08.085307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.085851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.086457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.090731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:08.093610 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:08.110947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1296/2523] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:08.135605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:08.151393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1988b-25ef-49fa-88df-42af97d375e1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:08.156998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:08.158693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:08.159267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:08.159591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] PATH_INFO: `/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e/users/d6ec50a600a04728bc683146aa1877ec/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:08.162803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44ac6e6134fd4cf49eb5ac830dbfa21e, user_id=d6ec50a600a04728bc683146aa1877ec, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:08.167992 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:08.185634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1228/2524] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:08 2022] PUT /identity/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e/users/d6ec50a600a04728bc683146aa1877ec/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:12.743897 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:12.767377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0821b-c885-4895-94fc-118039ba4c36 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:12.798279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:12.801806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:12.865713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:12.867067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:12.868185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:12.868687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] PATH_INFO: `/v3/users/f881c4404dba4b08ab64ed1f501542d4` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:12.870728 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f881c4404dba4b08ab64ed1f501542d4)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:12.874354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:12.925499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc66b326-20f6-4267-8706-c8f7c67c1ad5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:12.934454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:12.935364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:12.935894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:12.936167 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] PATH_INFO: `/v3/users/c743ef94052447bb8535cc13b328cfc6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:12.939092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c743ef94052447bb8535cc13b328cfc6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:12.942125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:12.960046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f881c4404dba4b08ab64ed1f501542d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:12.961489 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f881c4404dba4b08ab64ed1f501542d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:12.972081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f881c4404dba4b08ab64ed1f501542d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:12.988949 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f881c4404dba4b08ab64ed1f501542d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.005900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ef160bb7-b78f-4665-89d4-b686f47ced06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f881c4404dba4b08ab64ed1f501542d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.020821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1297/2525] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:12 2022] DELETE /identity/v3/users/f881c4404dba4b08ab64ed1f501542d4 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:13.026551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c743ef94052447bb8535cc13b328cfc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.028290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.028647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.028891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c743ef94052447bb8535cc13b328cfc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.029027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.033877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:13.061387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:13.062734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.064721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c743ef94052447bb8535cc13b328cfc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.079565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c743ef94052447bb8535cc13b328cfc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.095929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-b7d9aa3e-964c-4c8c-9f9d-df9304bb0e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c743ef94052447bb8535cc13b328cfc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.108525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1229/2526] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:12 2022] DELETE /identity/v3/users/c743ef94052447bb8535cc13b328cfc6 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:03:13.111124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.111124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.111819 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.119957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:13.140503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:13.141905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.172839 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1298/2527] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:13.196574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:13.198466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.298613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1230/2528] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:13.307169 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:13.307738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.325958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1f0f8-ec8d-4f72-bd84-2e100a8d320c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:13.386810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac4d88-0bfa-4564-a018-06c637c46dc3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:13.392653 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:13.393825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.393825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.394268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.404689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:13.406039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:13.406538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:13.406822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.407552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.407875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.423834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:13.425193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1231/2529] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:13.425796 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e12d-7b91-4778-a593-61f1c1541716 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:13.440166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1299/2530] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:03:13.581937 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:13.584987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.612051 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7b792-4cd9-4ce4-86d3-0b702b34a5a8 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:13.618816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:13.619524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.619823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.620104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] PATH_INFO: `/v3/users/351563957ef744bdb482bc9c303e05f2` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.621432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351563957ef744bdb482bc9c303e05f2)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:13.623489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:13.716336 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351563957ef744bdb482bc9c303e05f2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.718347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351563957ef744bdb482bc9c303e05f2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.758551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351563957ef744bdb482bc9c303e05f2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.761716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:13.773007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351563957ef744bdb482bc9c303e05f2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.788209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351563957ef744bdb482bc9c303e05f2'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:13.803600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1232/2531] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:03:13 2022] DELETE /identity/v3/users/351563957ef744bdb482bc9c303e05f2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:13.808231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:13.808744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:13.808744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:13.853774 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:13.855481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:13.941485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1300/2532] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:13.985210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:13.988631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:14.084120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10009d88-509d-4272-bcf3-512d72cab9d2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:14.090650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:14.091433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.091836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.092283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.108718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:14.114221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:14.132008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1233/2533] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:14.229064 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:14.230528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:14.244449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49fa0ea-a413-4841-b34a-851d901ff528 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:14.251081 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:14.252170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.252451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.252724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] PATH_INFO: `/v3/projects/a529db680ed84bd081e73459a45e388d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.254312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a529db680ed84bd081e73459a45e388d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:14.257257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:14.318957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:14.330073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:14.333351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:14.348599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a529db680ed84bd081e73459a45e388d', 'request_id': 'req-a7993b4b-673b-4e45-85bd-3136c415c2d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.365878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a529db680ed84bd081e73459a45e388d', 'request_id': 'req-a7993b4b-673b-4e45-85bd-3136c415c2d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.377449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a529db680ed84bd081e73459a45e388d', 'request_id': 'req-a7993b4b-673b-4e45-85bd-3136c415c2d4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.381035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699a4256-81e7-473f-9a72-59bfdddfe7ff service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:14.404677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] The token cache is being invalidate because project a529db680ed84bd081e73459a45e388d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:14.405426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.406121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a7993b4b-673b-4e45-85bd-3136c415c2d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.419820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1301/2534] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:14 2022] DELETE /identity/v3/projects/a529db680ed84bd081e73459a45e388d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:14.568063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:14.570461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.571066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.571569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] PATH_INFO: `/v3/projects/9b6413b980ac4937b6ddc13988736b24` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.574163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6413b980ac4937b6ddc13988736b24)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:14.580895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:14.684858 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6413b980ac4937b6ddc13988736b24', 'request_id': 'req-4e752fd0-405d-4b6f-9b33-8325acd69f0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.695843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6413b980ac4937b6ddc13988736b24', 'request_id': 'req-4e752fd0-405d-4b6f-9b33-8325acd69f0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.703628 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6413b980ac4937b6ddc13988736b24', 'request_id': 'req-4e752fd0-405d-4b6f-9b33-8325acd69f0f'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.725354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] The token cache is being invalidate because project 9b6413b980ac4937b6ddc13988736b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:14.727500 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.742447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4e752fd0-405d-4b6f-9b33-8325acd69f0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:14.749341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1234/2535] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:14 2022] DELETE /identity/v3/projects/9b6413b980ac4937b6ddc13988736b24 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:14.756040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.756469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.757108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:14.805904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:14.807225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:14.988360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1302/2536] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:14.997590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:14.998071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:14.999596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.044861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:15.053501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.064666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1235/2537] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:15.068853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.087604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3d3e7-b9a1-4331-8146-1e76600775d4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.093560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.094959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.095552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.095957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.096794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:03:15.097107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.099058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.118269 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1303/2538] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:03:15.122670 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.174024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.179524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5004b5d-0f58-4581-9891-f6412b8ec2b2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.180454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.199810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.205308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.205615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.206674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.207348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.215394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.232387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5f24e-d9d7-4e2f-8ee9-91faf123f652 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.238519 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:15.253812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.254816 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.258050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.258521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] PATH_INFO: `/v3/projects/32d23b54bac549a1843099b477ed8cd3` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.263447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32d23b54bac549a1843099b477ed8cd3)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.272618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.305341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1236/2539] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:03:15 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:03:15.309547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.404219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32d23b54bac549a1843099b477ed8cd3', 'request_id': 'req-101fd8ce-b30d-46fc-a979-81fffa7a3071'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:15.420521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32d23b54bac549a1843099b477ed8cd3', 'request_id': 'req-101fd8ce-b30d-46fc-a979-81fffa7a3071'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:15.437623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32d23b54bac549a1843099b477ed8cd3', 'request_id': 'req-101fd8ce-b30d-46fc-a979-81fffa7a3071'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:15.443495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc4eaf-2751-433c-8941-352a352b8ddc admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.456971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] The token cache is being invalidate because project 32d23b54bac549a1843099b477ed8cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:03:15.457300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:15.457962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:03:15.459829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1304/2540] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:03:15 2022] DELETE /identity/v3/projects/32d23b54bac549a1843099b477ed8cd3 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:15.543823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.546197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.546555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.546767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.553341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.555767 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.614133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1237/2541] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:03:15 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:03:15.615980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.626587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.681245 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101fd8ce-b30d-46fc-a979-81fffa7a3071 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.690426 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.691063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.691344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.691620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.692383 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.695012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.702480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.720107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.760066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1305/2542] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:15.761319 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.773382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127888f8-fb01-4c38-8081-01c1a0ad0be6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.798315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.798970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.799253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.800710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.804545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.806272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.837485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1306/2543] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:03:15.839685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:15.878770 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99d2f1-47f8-459a-a0d8-ac56d4091a4b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.883416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.884086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.884489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.884669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.887512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db249a-46c4-44d6-af5d-caf6ab002151 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:15.889301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:15.901910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:15.902623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.902906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.905017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] PATH_INFO: `/v3/projects/e8864aa3c8294fae86491fe3f3afc444/users/5d81e1aac9e14a77ac5007c9bb82b3e0/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.907816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.909776 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6a495-8dd2-4d6a-91f3-cff67ca0e1b4 service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.926087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1238/2544] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:15.928196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:15.943581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8864aa3c8294fae86491fe3f3afc444, user_id=5d81e1aac9e14a77ac5007c9bb82b3e0, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:15.947867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:15.967709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1307/2545] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:03:15 2022] PUT /identity/v3/projects/e8864aa3c8294fae86491fe3f3afc444/users/5d81e1aac9e14a77ac5007c9bb82b3e0/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:03:15.970887 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23907-89cf-498f-bcb4-6ed699c581a0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:15.971246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23907-89cf-498f-bcb4-6ed699c581a0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:15.971591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be23907-89cf-498f-bcb4-6ed699c581a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:15.995977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2be23907-89cf-498f-bcb4-6ed699c581a0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:15.997604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2be23907-89cf-498f-bcb4-6ed699c581a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.007821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:16.088249 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1239/2546] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:16.098322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:16.100122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.181390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792011a-fd0c-4a26-9a55-4d6b4e63766f admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:16.186250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:16.186896 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.187215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.187443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.198930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:16.200545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:16.232747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1308/2547] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:16.588026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e2c0-439a-44ea-a2e6-4b9a641310ca None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.588386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e2c0-439a-44ea-a2e6-4b9a641310ca None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.588672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814e2c0-439a-44ea-a2e6-4b9a641310ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.658618 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d814e2c0-439a-44ea-a2e6-4b9a641310ca None None] MFA Rules not processed for user `d6ec50a600a04728bc683146aa1877ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:16.663488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d814e2c0-439a-44ea-a2e6-4b9a641310ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.790771 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1240/2548] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:16.805229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:16.806917 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.894945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c05bed-96f6-4ded-a048-b061487babc4 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:16.902033 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:16.902871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:16.903151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:16.903451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:16.905667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:16.980039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:16.981805 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:16.997719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1309/2549] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:17.061313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9518c-9442-40e9-b74c-b3ded2fa5fad None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.062262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9518c-9442-40e9-b74c-b3ded2fa5fad None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.063300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9518c-9442-40e9-b74c-b3ded2fa5fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.113012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-13b9518c-9442-40e9-b74c-b3ded2fa5fad None None] MFA Rules not processed for user `5d81e1aac9e14a77ac5007c9bb82b3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:17.114334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-13b9518c-9442-40e9-b74c-b3ded2fa5fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.217708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1241/2550] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:17.230819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:17.232355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:17.305785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:17.363936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0bf8-c38d-4dc8-9f05-718ac2213b3a service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:17.369974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:17.370602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:17.370881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:17.371330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:17.382198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:17.383744 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca64dd-462a-45a5-8cdc-9a841afe0d81 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:17.396710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1310/2551] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:18.289172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86063c-44d2-4cab-8afd-ddbb2c5097af None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.289728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86063c-44d2-4cab-8afd-ddbb2c5097af None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.289952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d86063c-44d2-4cab-8afd-ddbb2c5097af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:18.318948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9d86063c-44d2-4cab-8afd-ddbb2c5097af None None] MFA Rules not processed for user `5f392617d7db4aba8511603ca19ae462`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:18.321955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9d86063c-44d2-4cab-8afd-ddbb2c5097af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:18.333482 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1242/2552] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:18.996804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:18.997363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:18.997363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:19.040798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] MFA Rules not processed for user `5d81e1aac9e14a77ac5007c9bb82b3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:19.042061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:19.059732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1311/2553] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:28.310689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:29.025847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=17 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:30.166806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c484-9d2e-41af-9c58-9f420a4a2f48 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:30.166806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c484-9d2e-41af-9c58-9f420a4a2f48 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:30.166806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17c484-9d2e-41af-9c58-9f420a4a2f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:30.186475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5e17c484-9d2e-41af-9c58-9f420a4a2f48 None None] MFA Rules not processed for user `a08509edf0f14a1c96035684290226f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:03:30.188642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5e17c484-9d2e-41af-9c58-9f420a4a2f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:30.277420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1243/2554] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:03:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:03:40.269813 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=25 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:03:48.744290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:03:48.745759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.757909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a56dbd8-17f7-46f1-a078-2ad9432fa9b8 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:03:48.761481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:03:48.762090 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:03:48.762371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:03:48.762644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:03:48.764649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:03:48.829235 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:03:48.830703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:03:48.840419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1312/2555] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:03:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:03:58.836113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=30 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:11.818894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:11.819437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:11.819547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:11.838665 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:11.839917 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:11.919537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1244/2556] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:04:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:11.968999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:11.971253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:12.071894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce966ad-fcf4-474b-98f2-c4cda401d04d service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:12.071894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:12.071894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:12.071894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:12.072503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:12.082075 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:12.083659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e2bd-def2-4415-aabf-5e4cf58e70c9 service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:12.092558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1313/2557] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:04:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:21.903416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=25 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:22.088574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=32 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:23.678561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:23.696412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6151ef5-9911-4ffe-95e6-0b01b8440b46 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:23.709393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:23.711539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:23.711852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:23.712229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:23.730176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:23.733010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:23.752853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1245/2558] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:32.263536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:32.264197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:32.264197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:32.295338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:32.296697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:32.311122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1314/2559] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:04:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:04:32.329835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:32.346219 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de9da9-dc30-4e96-a117-e3a801e65be3 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:32.352011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:32.352839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:32.353124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:32.353398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:32.371164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:32.371164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:32.377809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1246/2560] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:04:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:04:34.811730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:34.814119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:34.830102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e440e-760b-4adc-8d5a-b16b79558704 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:34.834691 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:34.835351 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:34.835826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:34.836036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] PATH_INFO: `/v3/users/a08509edf0f14a1c96035684290226f2` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:34.837140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a08509edf0f14a1c96035684290226f2)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:34.839736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:34.893497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a08509edf0f14a1c96035684290226f2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:34.894186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a08509edf0f14a1c96035684290226f2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:34.904365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a08509edf0f14a1c96035684290226f2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:34.919861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a08509edf0f14a1c96035684290226f2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:34.927528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-3753a3c5-2529-4912-892e-e33084e0c692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a08509edf0f14a1c96035684290226f2'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:34.934257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1315/2561] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:04:34 2022] DELETE /identity/v3/users/a08509edf0f14a1c96035684290226f2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:35.131584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:35.147370 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab7cf3-179f-410b-ae14-355c303d1989 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:35.223453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:35.224164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:35.224434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:35.224901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] PATH_INFO: `/v3/projects/7cc6df83bfd54df6b8c34a893e08faa1` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:35.226282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc6df83bfd54df6b8c34a893e08faa1)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:35.228268 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:35.307427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc6df83bfd54df6b8c34a893e08faa1', 'request_id': 'req-7fddfbb2-3be4-4e99-84ab-05c667c1004d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:35.322258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc6df83bfd54df6b8c34a893e08faa1', 'request_id': 'req-7fddfbb2-3be4-4e99-84ab-05c667c1004d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:35.334509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc6df83bfd54df6b8c34a893e08faa1', 'request_id': 'req-7fddfbb2-3be4-4e99-84ab-05c667c1004d'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:35.350351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] The token cache is being invalidate because project 7cc6df83bfd54df6b8c34a893e08faa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:04:35.350790 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:35.351573 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:04:35.354120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1247/2562] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:04:35 2022] DELETE /identity/v3/projects/7cc6df83bfd54df6b8c34a893e08faa1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:04:35.363846 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0744d66-0584-47ee-8754-c646332851ea None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:35.364249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0744d66-0584-47ee-8754-c646332851ea None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:35.364921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0744d66-0584-47ee-8754-c646332851ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:35.421755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-d0744d66-0584-47ee-8754-c646332851ea None None] MFA Rules not processed for user `cb8db3b650f24566b537fcb10e7f46ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:04:35.425521 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d0744d66-0584-47ee-8754-c646332851ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:35.506711 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1316/2563] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:04:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:04:45.345723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:45.498741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:04:55.773918 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:55.775539 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.837120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fddfbb2-3be4-4e99-84ab-05c667c1004d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:55.842129 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:55.842742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:55.843021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:55.843294 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:55.845048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.959335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0744d66-0584-47ee-8754-c646332851ea None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:04:55.962217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d0744d66-0584-47ee-8754-c646332851ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:55.966192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:55.970384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472d3fc-23bb-4b57-91f0-4545618a998b service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:55.998521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1248/2564] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:04:56.054598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0744d66-0584-47ee-8754-c646332851ea None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:04:56.059665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:04:56.060468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:04:56.060758 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:04:56.061036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:04:56.063058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:04:56.079934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:04:56.081422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:04:56.096845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1317/2565] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:04:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:05:05.984754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=54 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:06.088528 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=26 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:17.788434 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:17.788933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:17.789036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:17.810082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] MFA Rules not processed for user `5d81e1aac9e14a77ac5007c9bb82b3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:05:17.811302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:17.864910 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1249/2566] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:05:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:05:19.419507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:19.421025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:19.499329 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eca97c-00d5-41d0-bcf7-da6eb895e849 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:19.504397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:19.505002 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:19.505281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:19.505556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:19.507293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:19.519316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:19.520940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:19.531306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1318/2567] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:21.367649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:05:21.369846 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:21.381831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945f1f3-67e0-4951-9a7b-b4129d5d6c37 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:05:21.385369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:05:21.385969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:05:21.386242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:05:21.386507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:05:21.388209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:05:21.397423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:05:21.398845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:05:21.406851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1250/2568] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:05:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:05:29.527061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=32 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:05:31.403378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=31 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:02.851076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:02.852899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:02.870930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b9759-162b-44ed-9aa4-3f0e97040682 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:02.876442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:02.877050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:02.877354 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:02.877606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:02.879555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:02.893879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:02.895453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:02.906507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1319/2569] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:12.901906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:32.494958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:32.496574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:32.573803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df9100-f588-4040-9600-6110f6961efa service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:32.578903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:32.579727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:32.580088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:32.580512 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] PATH_INFO: `/v3/users/5d81e1aac9e14a77ac5007c9bb82b3e0` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:32.581723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d81e1aac9e14a77ac5007c9bb82b3e0)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:32.583669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:32.637034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d81e1aac9e14a77ac5007c9bb82b3e0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:32.637803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d81e1aac9e14a77ac5007c9bb82b3e0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:32.650938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d81e1aac9e14a77ac5007c9bb82b3e0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:32.661067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d81e1aac9e14a77ac5007c9bb82b3e0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:32.673313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d81e1aac9e14a77ac5007c9bb82b3e0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:32.683823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1251/2570] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:06:32 2022] DELETE /identity/v3/users/5d81e1aac9e14a77ac5007c9bb82b3e0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:32.875190 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:32.892220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ec62e-4515-4df0-bb60-a94755cc1df6 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:32.953865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:32.954483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:32.954764 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:32.955038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] PATH_INFO: `/v3/projects/e8864aa3c8294fae86491fe3f3afc444` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:32.956216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8864aa3c8294fae86491fe3f3afc444)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:32.958310 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:33.008944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8864aa3c8294fae86491fe3f3afc444', 'request_id': 'req-e52cd377-ae47-40e7-9eef-298239482ee4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:33.017819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8864aa3c8294fae86491fe3f3afc444', 'request_id': 'req-e52cd377-ae47-40e7-9eef-298239482ee4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:33.025714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8864aa3c8294fae86491fe3f3afc444', 'request_id': 'req-e52cd377-ae47-40e7-9eef-298239482ee4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:33.036988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] The token cache is being invalidate because project e8864aa3c8294fae86491fe3f3afc444 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:06:33.037282 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:33.037960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:06:33.040133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1320/2571] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:06:32 2022] DELETE /identity/v3/projects/e8864aa3c8294fae86491fe3f3afc444 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:06:42.677384 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:43.033654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:06:44.114087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:06:44.115658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:44.202286 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994bab8-40ba-48f7-a783-443b7e41dd3e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:06:44.269240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:06:44.269859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:06:44.270137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:06:44.270410 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:06:44.272207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:06:44.346495 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:06:44.348151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee82c3-efa0-46c8-8d0c-e74256cf2c9e service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:06:44.358431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1252/2572] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:06:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:06:54.353642 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=84 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:13.750732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:13.752625 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:13.769592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cd377-ae47-40e7-9eef-298239482ee4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:13.776342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:13.776959 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:13.777239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:13.777517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:13.779470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:13.842412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:13.844109 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:13.854003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1321/2573] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:07:23.849312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=28 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:37.557216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:37.557808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:37.557808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:37.579695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] MFA Rules not processed for user `cb8db3b650f24566b537fcb10e7f46ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:07:37.580964 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:37.588143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1253/2574] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:07:37 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:07:42.899892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:42.901454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:42.918669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade1ac2-6ddf-46ab-8971-2c23fe9a9cd2 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:42.925027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:42.925668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:42.926086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:42.926231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:42.928056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:42.942065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:42.943688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:42.956422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1322/2575] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:46.597950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:46.620295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5283211-3e65-494d-a4bf-27cd38929de3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:46.627971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:46.628991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:46.629277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:46.629788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:46.632227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:46.715932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:46.717475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:46.727393 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1254/2576] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:07:52.951095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:56.722930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=31 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:57.367833 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:57.477325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba11b76-a9ad-45ec-8221-ba1a488e5908 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:57.484647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:57.485827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:57.486510 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.486867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.487230 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.487634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.489876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.538185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:57.540202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d7f54-fae2-48a5-9253-2bf2412edf53 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:57.553608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1323/2577] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:57.596621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456dc9f-f020-4de4-bada-fc0d6fd244fe service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:57.600436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:57.601048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.601372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.601621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.603437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.614487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:57.615976 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5edd667-f649-4a9e-b763-b4c7778a6ce9 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:57.632142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1255/2578] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:57.700487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.701051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.701361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.757550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:07:57.759123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.781132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b3e7-6f6f-4eff-81ea-679e30580f4d None None] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.781682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b3e7-6f6f-4eff-81ea-679e30580f4d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.781682 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1b3e7-6f6f-4eff-81ea-679e30580f4d None None] PATH_INFO: `/` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.783111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1256/2579] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:07:57.790615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.790913 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.791392 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.837212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1324/2580] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:07:57.852470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] MFA Rules not processed for user `77e0d9362b3241d3b888e1cb0a2b5f4d`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:07:57.854674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.866040 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1257/2581] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:07:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:07:57.879550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:57.895987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b729a22-60e9-4271-a898-6d8a32329328 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:57.899503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:57.900295 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:57.901078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:57.901465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:57.901677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:57.901931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:57.913285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:57.915098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04f075-722c-47e8-b6ab-1d469e8649e2 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:57.925905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1325/2582] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:58.000337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf411-e322-4e11-8b08-2db00d3d78ed None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:58.026785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141217-238f-4510-bcf4-7fcb951751f9 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.027740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141217-238f-4510-bcf4-7fcb951751f9 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.028872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2141217-238f-4510-bcf4-7fcb951751f9 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.029491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:58.030133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.030413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.030684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.031745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1326/2583] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:07:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 10:07:58.038530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126deee-0062-49ec-966f-cda486049418 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.038811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126deee-0062-49ec-966f-cda486049418 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.039472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126deee-0062-49ec-966f-cda486049418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.040563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:58.042244 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:58.053684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1258/2584] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:07:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 10:07:58.064068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5126deee-0062-49ec-966f-cda486049418 None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:07:58.066398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5126deee-0062-49ec-966f-cda486049418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:58.091862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1327/2585] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:07:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:07:58.102646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:58.104275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:58.124574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc5033-31e8-48b0-8974-a3a7b230ca62 service swift] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:58.129158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:58.129793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.130139 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.130366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.142536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:58.144250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:58.157947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1259/2586] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:07:58.498953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126deee-0062-49ec-966f-cda486049418 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:07:58.509502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126deee-0062-49ec-966f-cda486049418 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:07:58.514162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:07:58.514886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:07:58.515251 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:07:58.515598 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:07:58.517522 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:07:58.582133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:07:58.625790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:07:58.627274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:07:58.651662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1328/2587] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:08:02.584189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:02.585633 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:02.647376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=100 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:02.723877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afda081-9f75-41b1-b701-bd35b85c2a0d service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:02.729494 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:02.730316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:02.730674 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:02.731005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:02.732958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:02.745657 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:02.747531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:02.759713 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1260/2588] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:06.197149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:06.209861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e5544b-579c-4b1f-9777-c5a7e927dade service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:06.222003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:06.222003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:06.222003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:06.222003 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:06.224207 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:06.240740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:06.242353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:06.264637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1329/2589] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:08:06.282545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:06.284242 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:06.299509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c92a8-9089-468a-8818-c21eaa8402ab service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:06.304227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:06.304907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:06.305185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:06.305442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] PATH_INFO: `/v3/users/21ac2c8187dc46aaa687e149bd9517b7` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:06.306541 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21ac2c8187dc46aaa687e149bd9517b7)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:06.308546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:06.367640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21ac2c8187dc46aaa687e149bd9517b7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.368284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21ac2c8187dc46aaa687e149bd9517b7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.386055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21ac2c8187dc46aaa687e149bd9517b7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.395247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21ac2c8187dc46aaa687e149bd9517b7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.405548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21ac2c8187dc46aaa687e149bd9517b7'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.412075 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1261/2590] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:08:06 2022] DELETE /identity/v3/users/21ac2c8187dc46aaa687e149bd9517b7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:06.604736 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:06.620464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bd073-7bd5-469b-9ccb-946d5b848cc7 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:06.688304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:06.688945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:06.689254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:06.689523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] PATH_INFO: `/v3/projects/1a0d20426a9a4c98b68ee87ebfe62623` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:06.690649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a0d20426a9a4c98b68ee87ebfe62623)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:06.692589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:06.749546 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a0d20426a9a4c98b68ee87ebfe62623', 'request_id': 'req-192f5a78-c4f0-45ac-9db2-859cf179e383'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.757454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a0d20426a9a4c98b68ee87ebfe62623', 'request_id': 'req-192f5a78-c4f0-45ac-9db2-859cf179e383'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.764966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a0d20426a9a4c98b68ee87ebfe62623', 'request_id': 'req-192f5a78-c4f0-45ac-9db2-859cf179e383'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.777265 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] The token cache is being invalidate because project 1a0d20426a9a4c98b68ee87ebfe62623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:08:06.777828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.778740 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.780371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1330/2591] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:08:06 2022] DELETE /identity/v3/projects/1a0d20426a9a4c98b68ee87ebfe62623 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:08:06.784714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:06.786156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:06.799745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3f5f4-5c4d-47a2-ab61-b8b003fc1b6e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:06.859047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:06.859851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:06.860130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:06.860403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] PATH_INFO: `/v3/users/5a483c3f4c4b4968bcd2d575b5929246` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:06.861568 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a483c3f4c4b4968bcd2d575b5929246)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:06.863515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:06.894873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:06.921137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a483c3f4c4b4968bcd2d575b5929246'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.921843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a483c3f4c4b4968bcd2d575b5929246'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.938422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a483c3f4c4b4968bcd2d575b5929246'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.951993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a483c3f4c4b4968bcd2d575b5929246'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.960824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-272f12d2-4517-45b8-9a48-f9afff1edea5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a483c3f4c4b4968bcd2d575b5929246'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:06.968886 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1262/2592] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:08:06 2022] DELETE /identity/v3/users/5a483c3f4c4b4968bcd2d575b5929246 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:07.242391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:07.243997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:07.258051 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f5a78-c4f0-45ac-9db2-859cf179e383 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:07.295055 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:07.326293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:07.326939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:07.327220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:07.327534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] PATH_INFO: `/v3/projects/07a316770d43428883ed16509c4c3cb5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:07.328703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07a316770d43428883ed16509c4c3cb5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:07.330671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:07.514861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07a316770d43428883ed16509c4c3cb5', 'request_id': 'req-007d182d-4144-49b6-84ed-862eaf68def4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:07.522951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07a316770d43428883ed16509c4c3cb5', 'request_id': 'req-007d182d-4144-49b6-84ed-862eaf68def4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:07.531001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07a316770d43428883ed16509c4c3cb5', 'request_id': 'req-007d182d-4144-49b6-84ed-862eaf68def4'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:07.541706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] The token cache is being invalidate because project 07a316770d43428883ed16509c4c3cb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:08:07.541992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:07.542640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:08:07.544944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1331/2593] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:08:07 2022] DELETE /identity/v3/projects/07a316770d43428883ed16509c4c3cb5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:08:13.969146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:13.969671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:13.969671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:13.990486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] MFA Rules not processed for user `b99bdb65eb5c472ab7711735f3d35a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:08:13.991715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:14.049933 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1263/2594] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:08:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:08:17.538632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:19.300326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:08:19.302076 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:19.377642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d182d-4144-49b6-84ed-862eaf68def4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:08:19.382834 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:08:19.383472 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:08:19.383751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:08:19.384061 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:08:19.385892 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:08:19.398530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:08:19.400188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:08:19.413701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1332/2595] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:08:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:08:24.042046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=44,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:08:29.407429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=32 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:07.372840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:07.377606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:07.463243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da8219-c702-4b65-9bb6-d7216105e55c None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:07.467427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:07.468069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:07.468349 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:07.468621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:07.470428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:07.545717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:07.547307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:07.557705 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1264/2596] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:11.087126 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:11.089484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:11.112234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9392a4b-f37c-498a-aba2-5aca3f7b4298 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:11.117579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:11.118236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.118504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.118773 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] PATH_INFO: `/v3/users/cb8db3b650f24566b537fcb10e7f46ff` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.119967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb8db3b650f24566b537fcb10e7f46ff)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:11.121974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:11.187260 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb8db3b650f24566b537fcb10e7f46ff'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.187914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb8db3b650f24566b537fcb10e7f46ff'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.197290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb8db3b650f24566b537fcb10e7f46ff'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.214400 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb8db3b650f24566b537fcb10e7f46ff'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.227367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0df22d30-7508-4c40-9bfb-6ca9cf315406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb8db3b650f24566b537fcb10e7f46ff'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.233868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1333/2597] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:09:11 2022] DELETE /identity/v3/users/cb8db3b650f24566b537fcb10e7f46ff => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:09:11.444758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:11.461299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13e3f0e-328a-4a60-827c-f6c1e3721777 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:11.538122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:11.538760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.539042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.539314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] PATH_INFO: `/v3/projects/27be5464dad640c28245de15564dd256` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.540467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=27be5464dad640c28245de15564dd256)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:11.542386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:11.582456 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:11.598270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27be5464dad640c28245de15564dd256', 'request_id': 'req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.605380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27be5464dad640c28245de15564dd256', 'request_id': 'req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.612605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27be5464dad640c28245de15564dd256', 'request_id': 'req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.626754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] The token cache is being invalidate because project 27be5464dad640c28245de15564dd256 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:09:11.627144 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.627916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-9e9d88e6-8259-417d-a78a-e80b10b7ce6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:09:11.629598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1265/2598] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:09:11 2022] DELETE /identity/v3/projects/27be5464dad640c28245de15564dd256 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:09:11.635439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.635723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.636046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.688024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:11.689276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:11.754206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1334/2599] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:11.762927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.763215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.763546 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.785859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:11.787082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:11.796544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1266/2600] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:11.800967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:11.811046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a28b52a-21f4-4aca-95cd-c6909c021f2f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:11.815672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:11.816328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.816602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.816866 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.817513 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:09:11.817813 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:11.819416 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:11.831580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1335/2601] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:09:11.836612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:11.847428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c851eb-4432-4408-afcf-fc94695f86c7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:11.852388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:11.853002 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.853281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.853554 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.854220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:11.856163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:11.886454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1267/2602] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:09:11 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:09:11.890785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:11.901710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd2c22-8ff8-45d5-8f3e-f91d34475c20 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:11.943287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:11.970967 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:11.971583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:11.971912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:11.972162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:11.972808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:11.974737 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:12.008094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1336/2603] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:09:11 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:09:12.012271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:12.023930 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc71680-f12c-4dca-8095-f125b3d886b1 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:12.029673 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:12.030409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.030811 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.031089 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.031727 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:12.033409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:12.046010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1268/2604] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:09:12.051537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:12.067266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafed882-eafc-4e64-914f-1f3c2102b238 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:12.073004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:12.073784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.074066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.074333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.074981 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:12.077325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0ad3a-8fbf-4312-930b-3b0326020bfe admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:12.088879 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1337/2605] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:09:12.093954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:12.105391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91d7c7-24d1-463d-8204-81b2c2d2b0c6 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:12.111906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:12.112729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.113128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.113504 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] PATH_INFO: `/v3/projects/b5536163b82c4862a27be2f828556ac0/users/4f0825af3ebe40e29e4740d1ff7f509b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.131063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5536163b82c4862a27be2f828556ac0, user_id=4f0825af3ebe40e29e4740d1ff7f509b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:12.133780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:12.150221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1269/2606] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:09:12 2022] PUT /identity/v3/projects/b5536163b82c4862a27be2f828556ac0/users/4f0825af3ebe40e29e4740d1ff7f509b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:09:12.156756 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc25c-43e3-4fb7-8f23-4bbcce174cab None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.156926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc25c-43e3-4fb7-8f23-4bbcce174cab None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.157226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560cc25c-43e3-4fb7-8f23-4bbcce174cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.177962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-560cc25c-43e3-4fb7-8f23-4bbcce174cab None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:12.179180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-560cc25c-43e3-4fb7-8f23-4bbcce174cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:12.243263 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1338/2607] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:12.253730 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:12.255558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:12.317420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63068637-b277-40a2-9e45-eb4d8c84258b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:12.320993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:12.321601 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:12.321881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:12.322162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:12.335769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:12.337263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:12.347265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1270/2608] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:16.725525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e09b-af36-4080-9593-2cf929bbdc05 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:16.726039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e09b-af36-4080-9593-2cf929bbdc05 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:16.726039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47e09b-af36-4080-9593-2cf929bbdc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:16.759010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-5a47e09b-af36-4080-9593-2cf929bbdc05 None None] MFA Rules not processed for user `4f0825af3ebe40e29e4740d1ff7f509b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:16.760427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5a47e09b-af36-4080-9593-2cf929bbdc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:16.860887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1339/2609] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:09:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:09:16.883855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:16.885415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:16.959024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:16.980499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32b468-fcb6-432f-b21f-afac88315bdf service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:16.987226 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:16.988299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:16.988615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:16.988929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:16.999506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:17.002321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:17.013428 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1271/2610] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:18.548342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:18.548855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:18.548950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:18.574443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] MFA Rules not processed for user `4f0825af3ebe40e29e4740d1ff7f509b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:09:18.575881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:18.584278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1340/2611] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:09:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:09:18.942098 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:18.953082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54977054-9ce4-43b1-bbce-ff97e2aad1eb service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:18.957052 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:18.957664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:18.957940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:18.958209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:18.960083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:18.971847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:18.973192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:18.982836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1272/2612] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:28.564986 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:28.978082 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=13 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:09:50.940138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:50.941784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:50.956950 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8454ab8-9ab7-4909-b42c-1a881354619c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:50.961082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:50.961653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:50.961932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:50.962242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:50.963983 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:51.025857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:51.027307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:51.036751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1341/2613] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:55.769828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:55.778957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7be75c-fd64-4ff8-88bb-8b3833cbbb5d service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:55.782839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:55.783460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:55.783742 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:55.784045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:55.785839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:55.800726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:55.802159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:55.814530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1273/2614] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:57.125440 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:57.133858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bbdbb-97e5-4958-a6ec-fb017ed467e0 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:57.137614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:57.138232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:57.138510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:57.138784 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:57.141413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:57.154034 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:57.155863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:57.166206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1342/2615] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:09:59.110822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:09:59.112703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:59.202220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0628d53-e001-483e-8d4c-3b0d0670181f service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:09:59.210583 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:09:59.210843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:09:59.211057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:09:59.211338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:09:59.215417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:09:59.232027 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:09:59.234447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:09:59.248299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1274/2616] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:09:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:04.158926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:04.160600 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:04.178293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d21f6-a975-48e7-b504-4aeddc5f9938 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:04.182822 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:04.183444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.183724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.184070 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] PATH_INFO: `/v3/users/5f392617d7db4aba8511603ca19ae462` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.185504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f392617d7db4aba8511603ca19ae462)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:04.188289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:04.239299 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f392617d7db4aba8511603ca19ae462'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.239988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f392617d7db4aba8511603ca19ae462'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.247402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f392617d7db4aba8511603ca19ae462'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.259371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f392617d7db4aba8511603ca19ae462'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.266715 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f392617d7db4aba8511603ca19ae462'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.272567 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1343/2617] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:10:04 2022] DELETE /identity/v3/users/5f392617d7db4aba8511603ca19ae462 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:04.461485 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:04.472585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e839f-ed2d-422f-a4f1-caac35b4474e service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:04.533831 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:04.534367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.534647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.534920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] PATH_INFO: `/v3/projects/a7a284ee619946b28f3e70a1aa8d2662` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.536079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a284ee619946b28f3e70a1aa8d2662)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:04.538001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:04.584496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a284ee619946b28f3e70a1aa8d2662', 'request_id': 'req-ada392e5-d45a-43d2-b613-bd9df9e61d7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.593977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a284ee619946b28f3e70a1aa8d2662', 'request_id': 'req-ada392e5-d45a-43d2-b613-bd9df9e61d7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.600311 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a284ee619946b28f3e70a1aa8d2662', 'request_id': 'req-ada392e5-d45a-43d2-b613-bd9df9e61d7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.607437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:04.612238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] The token cache is being invalidate because project a7a284ee619946b28f3e70a1aa8d2662 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:10:04.612655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.613951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.615486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1275/2618] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:10:04 2022] DELETE /identity/v3/projects/a7a284ee619946b28f3e70a1aa8d2662 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:04.619616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:04.621138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:04.634085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3c74-69c3-42bf-800d-8c25975f41ee admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:04.684032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:04.690663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:04.691500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:04.692044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:04.692473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] PATH_INFO: `/v3/users/d6ec50a600a04728bc683146aa1877ec` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:04.694084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ec50a600a04728bc683146aa1877ec)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:04.696198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:04.744029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ec50a600a04728bc683146aa1877ec'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.744679 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ec50a600a04728bc683146aa1877ec'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.752468 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ec50a600a04728bc683146aa1877ec'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.764249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ec50a600a04728bc683146aa1877ec'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.770976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ec50a600a04728bc683146aa1877ec'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:04.776777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1344/2619] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:10:04 2022] DELETE /identity/v3/users/d6ec50a600a04728bc683146aa1877ec => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:10:04.961133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:04.962715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:04.976726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada392e5-d45a-43d2-b613-bd9df9e61d7e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:05.034736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:05.035379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:05.035659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:05.035968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] PATH_INFO: `/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:05.037102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44ac6e6134fd4cf49eb5ac830dbfa21e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:05.039029 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:05.085080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44ac6e6134fd4cf49eb5ac830dbfa21e', 'request_id': 'req-a3651a5b-072c-40be-a4c7-445c7e2d1b06'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:05.091915 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44ac6e6134fd4cf49eb5ac830dbfa21e', 'request_id': 'req-a3651a5b-072c-40be-a4c7-445c7e2d1b06'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:05.098698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44ac6e6134fd4cf49eb5ac830dbfa21e', 'request_id': 'req-a3651a5b-072c-40be-a4c7-445c7e2d1b06'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:05.109702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] The token cache is being invalidate because project 44ac6e6134fd4cf49eb5ac830dbfa21e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:10:05.109986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:05.110634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:05.112164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1276/2620] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:10:04 2022] DELETE /identity/v3/projects/44ac6e6134fd4cf49eb5ac830dbfa21e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:14.773605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=27,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:15.105954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:21.609642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:21.611134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:21.674531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1c07f-1bba-4d38-8df2-232df0b5ed2d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:21.724036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:21.724653 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:21.724933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:21.725209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:21.726994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:21.793745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:21.795210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:21.806307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1345/2621] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:25.183022 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:25.184576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:25.198925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651a5b-072c-40be-a4c7-445c7e2d1b06 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:25.204637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:25.205272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.205550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.205823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] PATH_INFO: `/v3/users/b99bdb65eb5c472ab7711735f3d35a60` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.221431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99bdb65eb5c472ab7711735f3d35a60)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:25.223424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:25.271835 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99bdb65eb5c472ab7711735f3d35a60'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.272490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99bdb65eb5c472ab7711735f3d35a60'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.285791 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99bdb65eb5c472ab7711735f3d35a60'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.293418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99bdb65eb5c472ab7711735f3d35a60'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.301334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-eb65f9f8-a47e-41fe-a712-35e97c8e61fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99bdb65eb5c472ab7711735f3d35a60'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.307271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1277/2622] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:10:25 2022] DELETE /identity/v3/users/b99bdb65eb5c472ab7711735f3d35a60 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:25.496276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:25.508831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e58d3c-1524-44a4-ab96-758351af1839 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:25.576413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:25.577046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.577429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.578218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] PATH_INFO: `/v3/projects/9a2a3741defe4613bc01a88bce3c0bf7` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.587250 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=100 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:25.591078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a2a3741defe4613bc01a88bce3c0bf7)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:25.593512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:25.646551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a2a3741defe4613bc01a88bce3c0bf7', 'request_id': 'req-441be625-1c89-45d4-9708-3f63608cc056'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.653705 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a2a3741defe4613bc01a88bce3c0bf7', 'request_id': 'req-441be625-1c89-45d4-9708-3f63608cc056'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.661883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a2a3741defe4613bc01a88bce3c0bf7', 'request_id': 'req-441be625-1c89-45d4-9708-3f63608cc056'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.673548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] The token cache is being invalidate because project 9a2a3741defe4613bc01a88bce3c0bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:10:25.673808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.674405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-441be625-1c89-45d4-9708-3f63608cc056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:10:25.675905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1346/2623] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:10:25 2022] DELETE /identity/v3/projects/9a2a3741defe4613bc01a88bce3c0bf7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:25.680931 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.681355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.681639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.701984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:25.703509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:25.763171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1278/2624] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:25.771809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.772107 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.772384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.792508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:25.793733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:25.802819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1347/2625] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:25.807148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:25.819012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f058d24-9815-4486-b4f8-d7a4d7c7b43b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:25.823625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:25.824259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.824534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.824803 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.825443 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:10:25.825740 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:25.827445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:25.840092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1279/2626] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:10:25.844725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:25.854407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a5099-7508-442f-bcde-7c60544c61b1 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:25.860296 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:25.860913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:25.861191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:25.861460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:25.862088 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:25.864021 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:25.905669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1348/2627] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:10:25 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:10:25.909971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:25.920250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ee39c-5689-41c2-9b3e-7f2049924334 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:25.991564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:26.007948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:26.008574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.008853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.009128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.009775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:26.011829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:26.040039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1280/2628] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:10:25 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:10:26.042318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:26.054108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb50a0-0dab-485f-ba1f-f5786150c021 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:26.058987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:26.059642 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.059962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.060236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.060875 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:26.063335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:26.080240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1349/2629] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:10:26.084303 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:26.095132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649bf1e-2468-468e-bb69-855c2501a05f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:26.100024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:26.100644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.100929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.101205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.101853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:26.103458 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab311bd2-3352-44b2-9a25-3e3df2adf837 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:26.238321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1281/2630] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:10:26.242754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:26.254257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786bf404-9bee-4871-9b35-544b68a44566 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:26.259134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:26.259808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.260096 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.260369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] PATH_INFO: `/v3/projects/f82591f81f494cb2b76ead9e406e5b56/users/1eb93e96417d4495b3577782be50bcbe/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.275509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82591f81f494cb2b76ead9e406e5b56, user_id=1eb93e96417d4495b3577782be50bcbe, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:26.278180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:26.293530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1350/2631] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:10:26 2022] PUT /identity/v3/projects/f82591f81f494cb2b76ead9e406e5b56/users/1eb93e96417d4495b3577782be50bcbe/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:10:26.299557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cde8c-eb12-42f9-bfbc-d46603614944 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.299877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cde8c-eb12-42f9-bfbc-d46603614944 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.300156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539cde8c-eb12-42f9-bfbc-d46603614944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.319186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-539cde8c-eb12-42f9-bfbc-d46603614944 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:26.320441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-539cde8c-eb12-42f9-bfbc-d46603614944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:26.381832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1282/2632] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:26.391926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:26.393640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:26.451290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aaec48-23b8-40ad-be6c-d48979f63aab admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:26.455675 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:26.456495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:26.456780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:26.457056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:26.467449 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:26.469825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:26.480215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1351/2633] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:30.748374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aa0a7-507e-4979-aac3-5f2d5b44cb8a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:30.748960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aa0a7-507e-4979-aac3-5f2d5b44cb8a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:30.748960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5aa0a7-507e-4979-aac3-5f2d5b44cb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:30.778307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-4c5aa0a7-507e-4979-aac3-5f2d5b44cb8a None None] MFA Rules not processed for user `1eb93e96417d4495b3577782be50bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:30.779570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4c5aa0a7-507e-4979-aac3-5f2d5b44cb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:30.849457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1283/2634] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:10:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:30.871136 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:30.872820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:30.896849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:30.969344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4429bd33-3971-403b-902d-6d5816e3cf3f service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:30.974798 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:30.975408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:30.975686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:30.975992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:30.984641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:30.995931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:30.995931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1352/2635] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:10:32.308492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:32.309121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:32.309121 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:32.329888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] MFA Rules not processed for user `1eb93e96417d4495b3577782be50bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:10:32.331116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:32.338048 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1284/2636] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:10:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:10:33.051874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:33.053435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:33.133537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f50cf-2e72-4954-971d-bdd5047c3cfb service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:33.138455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:33.139068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:33.139347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:33.139620 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:33.141537 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:33.153704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:33.155213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c2673-b7bd-4c38-9161-b9aeb7ac3849 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:33.166848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1353/2637] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:35.135469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:10:35.146703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a546464-8dae-4211-9e71-ed98f3d3f6f9 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:10:35.150295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:10:35.151223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:10:35.151499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:10:35.151764 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:10:35.153580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:10:35.165078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:10:35.166631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:10:35.176237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1285/2638] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:10:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:10:43.162881 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=58 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:10:45.172212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:05.992633 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8125-5e13-4e10-ad3d-150dafdc1e31 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:05.993168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8125-5e13-4e10-ad3d-150dafdc1e31 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:05.993284 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22c8125-5e13-4e10-ad3d-150dafdc1e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:06.052657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-e22c8125-5e13-4e10-ad3d-150dafdc1e31 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:06.053920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e22c8125-5e13-4e10-ad3d-150dafdc1e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:06.124724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1354/2639] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:11:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:06.181313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:06.182802 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:06.291267 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f166167-0474-41f3-800e-8841fff79f4e service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:06.296859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:06.298063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:06.298438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:06.298667 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:06.307591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:06.309231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696a836-52c2-4e48-960e-3b580c41ef1b service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:06.318551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1286/2640] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:11:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:11:16.121769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=40 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:16.315178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=41 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:19.732058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:19.732867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:19.732867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] PATH_INFO: `/` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:19.733964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1355/2641] 10.222.0.78 () {54 vars in 921 bytes} [Wed Aug 24 10:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 10:11:19.739637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ef52-fb3f-4827-b74a-e601d728785c None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:19.740239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ef52-fb3f-4827-b74a-e601d728785c None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:19.740239 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1ef52-fb3f-4827-b74a-e601d728785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:19.783218 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-afe1ef52-fb3f-4827-b74a-e601d728785c None None] MFA Rules not processed for user `7f645f811e804835a793f372a1ac7111`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:19.784501 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-afe1ef52-fb3f-4827-b74a-e601d728785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:19.796906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1287/2642] 10.222.0.78 () {58 vars in 1018 bytes} [Wed Aug 24 10:11:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:11:19.808993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:19.810751 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:19.915465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414ce39-ab09-40d2-8de0-d429a4a30524 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:19.919121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:19.919732 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:19.920039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:19.920311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:19.932057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:19.933508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:19.946078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1356/2643] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:11:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:11:22.592490 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6233-dd93-4f92-937d-d644fd694155 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:22.592997 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6233-dd93-4f92-937d-d644fd694155 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:22.593095 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf6233-dd93-4f92-937d-d644fd694155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:22.612591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-98cf6233-dd93-4f92-937d-d644fd694155 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:22.613804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-98cf6233-dd93-4f92-937d-d644fd694155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:22.621368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1288/2644] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:11:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:22.633973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:22.646703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85836fc4-e42c-4f42-b924-f763d6c319a2 service glance] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:22.651616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:22.652411 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:22.652703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:22.652975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:22.663121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:22.664745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:22.674432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1357/2645] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:11:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:11:32.606977 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=12 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:32.669990 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=41 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:49.341943 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831330-35c1-4327-92b9-2fc2d89e71a4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:49.342420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831330-35c1-4327-92b9-2fc2d89e71a4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:49.342529 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24831330-35c1-4327-92b9-2fc2d89e71a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:49.361679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-24831330-35c1-4327-92b9-2fc2d89e71a4 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:11:49.362861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-24831330-35c1-4327-92b9-2fc2d89e71a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:11:49.370278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1289/2646] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:11:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:11:49.382618 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:11:49.393760 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af865013-ac8f-4f5f-8a62-00e237b571a3 service swift] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:11:49.398963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:11:49.399561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:11:49.399867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:11:49.400140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:11:49.408391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:11:49.409956 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:11:49.417898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1358/2647] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:11:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:11:59.356694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:11:59.414757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:21.833081 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81622-3619-47cb-b6f6-55b1645b444e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:21.833889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81622-3619-47cb-b6f6-55b1645b444e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:21.833889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b81622-3619-47cb-b6f6-55b1645b444e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:21.854283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-52b81622-3619-47cb-b6f6-55b1645b444e None None] MFA Rules not processed for user `1eb93e96417d4495b3577782be50bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:21.855491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-52b81622-3619-47cb-b6f6-55b1645b444e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:21.862400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1290/2648] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:12:25.460243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:12:25.475336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30709aa1-617b-4bb6-842f-62c247ee1df3 service swift] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:12:25.481185 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:12:25.481827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:25.482114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:25.482395 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:25.484242 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:25.498706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:12:25.500353 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:12:25.512287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1359/2649] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:12:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:12:31.849559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:35.507286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:12:52.812962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:12:52.813542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:12:52.813542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:12:52.835429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] MFA Rules not processed for user `4f0825af3ebe40e29e4740d1ff7f509b`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:12:52.836687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:12:52.891754 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1291/2650] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:12:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:13:01.470485 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:01.484891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd2090-222a-4e95-826b-0cfc8ab7a82d service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:01.490854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:01.491464 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:01.491755 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:01.492041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:01.493785 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:01.509283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:01.510951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:01.523164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1360/2651] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:02.885251 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=25 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:05.624116 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:13:05.625893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:05.643587 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a620ac9-2b8c-48fb-94c3-f2afb9bf150b None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:13:05.650288 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:13:05.650938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:13:05.651224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:13:05.651502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:13:05.653444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:13:05.667629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:13:05.669301 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e554049-59b8-42a1-8df5-287cdcada411 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:13:05.681548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1292/2652] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:13:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:13:11.517363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:13:15.676071 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=6 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:04.563220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:04.564726 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:04.581258 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddeb3ae-37f3-4a7b-aa6f-6e421e7a5125 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:04.587488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:04.588135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:04.588617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:04.588890 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:04.590659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:04.604106 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:04.606052 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:04.618964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1361/2653] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:13.166342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:13.166884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:13.166995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:13.187213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] MFA Rules not processed for user `1eb93e96417d4495b3577782be50bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:14:13.188438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:13.195678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1293/2654] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:14:14.613709 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:23.182404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:36.134942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:36.147226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f4681a-6e0f-4f17-a5b4-0f7f653ab590 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:36.153135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:36.153731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:36.154012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:36.154283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:36.156030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:36.166542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:36.168164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:36.176992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1362/2655] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:46.172946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:14:56.164373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:56.174880 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6d429-ae31-4ea1-8c0d-528a6f2f8a72 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:56.178655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:56.179246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:56.179526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:56.179824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:56.181700 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:56.195655 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:56.197181 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d93d3-f539-4916-8f16-dd10114f968e service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:56.208072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1294/2656] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:14:59.768178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:14:59.769664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:59.781686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebf43a-cf15-4478-9b37-853627c9b01e service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:14:59.785266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:14:59.785854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:14:59.786134 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:14:59.786404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:14:59.788083 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:14:59.801587 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:14:59.803006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:14:59.813878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1363/2657] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:14:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:01.141778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:01.142492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:01.142492 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:01.163105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:01.164351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:01.173057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1295/2658] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:01.218412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:01.230325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da42eac5-2669-413e-84e7-b5f5a0337ad3 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:01.235304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:01.235928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:01.236210 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:01.236570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:01.249909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:01.252252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:01.261926 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1364/2659] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:15:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:15:11.158140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=9 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:11.259241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=12 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:20.102952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:20.113724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e614dd-5f6e-4984-a27d-df52aadc2190 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:20.117320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:20.118137 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:20.118462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:20.118907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:20.121365 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:20.132716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:20.134376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa24d6-2157-4581-b52c-869870257bfe service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:20.143758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1296/2660] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:20.228557 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:20.241731 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6233b27-22aa-4cf0-ba14-e6d1de614b05 service swift] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:20.245971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:20.246688 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:20.246971 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:20.247252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:20.249094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:20.263668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:20.265180 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:20.277434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1365/2661] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:21.929712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:21.930209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:21.930321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:21.950906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:21.952166 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:21.961889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1297/2662] 10.222.0.78 () {58 vars in 1009 bytes} [Wed Aug 24 10:15:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:21.976482 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:21.989831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d602d-f3dd-43f5-89e6-8a9f4230bb60 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:21.995170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:21.995871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:21.996270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:21.996660 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:22.005441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:22.007040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c700e-54fe-4529-bb3b-0421adf61fdf service swift] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:22.016013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1366/2663] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:15:31.944854 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=9 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:32.012769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=12 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:42.512701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:42.522508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a81ff93-2b9b-4883-9e72-9ed14e8119e7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:42.526047 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:42.526639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:42.527430 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:42.527903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:42.535168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:42.536634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:42.544640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1298/2664] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:15:44.700050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:44.700534 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:44.700649 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:44.720049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] MFA Rules not processed for user `b5793768f3fe46ab8b16908d5f570c91`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:15:44.721248 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:15:44.728372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1367/2665] 10.222.0.78 () {58 vars in 967 bytes} [Wed Aug 24 10:15:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:15:44.740960 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:15:44.753450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc9c5dd-83cf-49ec-96d7-80b04dddcb49 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:15:44.758324 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61f9de1f0b754d74bf44b92eb2134081', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:15:44.759170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:15:44.759454 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:15:44.759722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:15:44.769423 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:15:44.771028 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928e56a-11f0-446a-a236-daa447b015be service swift] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:15:44.779160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1299/2666] 10.222.0.78 () {60 vars in 1384 bytes} [Wed Aug 24 10:15:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 10:15:54.715157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:15:54.775778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=12 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:21.323750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:21.337483 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d098f-8a43-40a4-81b3-5a2ab68ca993 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:21.344068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:21.344662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:21.344936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:21.345205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:21.353011 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:21.354580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:21.364476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1368/2667] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:31.360657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:38.670321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:38.670888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:38.670888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:38.690205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] MFA Rules not processed for user `1eb93e96417d4495b3577782be50bcbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:16:38.691407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:38.698221 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1300/2668] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:16:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:16:39.003923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:39.013558 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f683729-1f95-4c34-ac0b-7929360d7acd service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:39.017277 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:39.018009 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:39.018300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:39.018566 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:39.020347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:16:39.030341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:39.031829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:39.040542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1369/2669] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:41.372152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:16:41.385201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc29724-efd8-40f1-a5c2-6d50c6d9debc None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:16:41.390343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:16:41.390935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:16:41.391209 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:16:41.391478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:16:41.401091 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:16:41.402898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:16:41.412720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1301/2670] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:16:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:16:49.036923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:16:51.408953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=9 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:01.677536 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:01.686252 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c136340-6b9f-4d5b-8e84-b507033275ae service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:01.689904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:01.690520 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:01.690799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:01.691145 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:01.692877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:01.704488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:01.705994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:01.715886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1370/2671] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:04.854649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.865851 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb38f6-af04-42ce-8eed-d692880f9d4f service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.871062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.871733 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.872109 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.872469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.874424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:04.887678 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.889322 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:04.902649 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1302/2672] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:04.919713 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:04.921856 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:04.937267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a8c5e-e900-42f5-be47-5ce80af1ec94 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:04.942519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:04.943390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:04.943858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:04.944300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] PATH_INFO: `/v3/users/4f0825af3ebe40e29e4740d1ff7f509b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:04.945565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0825af3ebe40e29e4740d1ff7f509b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:04.947999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.002639 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0825af3ebe40e29e4740d1ff7f509b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.003454 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0825af3ebe40e29e4740d1ff7f509b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.014198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0825af3ebe40e29e4740d1ff7f509b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.027172 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0825af3ebe40e29e4740d1ff7f509b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.036560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-57d6ab01-d29e-4b33-95b3-2af57fa4487b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0825af3ebe40e29e4740d1ff7f509b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.043017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1371/2673] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:04 2022] DELETE /identity/v3/users/4f0825af3ebe40e29e4740d1ff7f509b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:05.287009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.297619 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a59b-86aa-43ad-ac6d-977d0bb40f75 service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.357427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.358061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.358359 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.358644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] PATH_INFO: `/v3/projects/b5536163b82c4862a27be2f828556ac0` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.359745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5536163b82c4862a27be2f828556ac0)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.361677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.410080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5536163b82c4862a27be2f828556ac0', 'request_id': 'req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.417295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5536163b82c4862a27be2f828556ac0', 'request_id': 'req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.423857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5536163b82c4862a27be2f828556ac0', 'request_id': 'req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.435224 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] The token cache is being invalidate because project b5536163b82c4862a27be2f828556ac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:05.435497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.437266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-785b1870-b6a1-4e5f-bd61-bf54c0abe7ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:05.438760 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1303/2674] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:05 2022] DELETE /identity/v3/projects/b5536163b82c4862a27be2f828556ac0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:05.444480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321523f6-9663-47dc-b015-0db249977e8d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.444775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321523f6-9663-47dc-b015-0db249977e8d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.445050 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321523f6-9663-47dc-b015-0db249977e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.465223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-321523f6-9663-47dc-b015-0db249977e8d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.466435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-321523f6-9663-47dc-b015-0db249977e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.526577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1372/2675] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.534808 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.535076 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.535354 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.557414 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.558646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.567638 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1304/2676] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.571976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321523f6-9663-47dc-b015-0db249977e8d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.582547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321523f6-9663-47dc-b015-0db249977e8d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.587186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.587815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.588110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.588378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.589025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:05.589325 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.590909 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.603434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1373/2677] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:17:05.607784 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.617085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d7b91-3ea9-493f-9d96-9a1382ef5337 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.621750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.622347 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.622634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.622907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.623526 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.625460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.654738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1305/2678] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:05.658635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.669309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607a5f-01e4-40fd-81cc-16ad0a04bcbf admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.715169 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:05.730616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.731225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.731507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.731783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.732451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.734446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.758387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1374/2679] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:05.762429 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.773621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b38b5-ffcf-4de2-af3b-f9f4f9e5e5fb admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.778379 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.779008 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.779270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.779542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.780332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.781979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.792996 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1306/2680] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:05.796976 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.807614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2440e37-c6f0-4af8-9cbd-ec468b0bb041 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.812264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.812885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.813166 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.813439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.814078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.816749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd397708-b414-4a65-8401-61df193f0b7f admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.827397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1375/2681] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:05.831605 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:05.841565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a1066-b319-46ae-9bca-1daaefc6fdf8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:05.846381 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:05.847171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.847544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.848057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] PATH_INFO: `/v3/projects/c8364f16d5d34028be237c349e2bb88c/users/29e62588df6649fcb2b8b382ad638f52/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.864815 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8364f16d5d34028be237c349e2bb88c, user_id=29e62588df6649fcb2b8b382ad638f52, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:05.867481 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:05.887014 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1307/2682] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:05 2022] PUT /identity/v3/projects/c8364f16d5d34028be237c349e2bb88c/users/29e62588df6649fcb2b8b382ad638f52/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:05.899149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:05.899433 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:05.899702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:05.927453 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] MFA Rules not processed for user `29e62588df6649fcb2b8b382ad638f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:05.928697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:05.993013 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1376/2683] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:05.997424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.011278 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:06.061274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71d87cc-fd00-4dc3-819e-5a6000ae97e2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.065983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.066580 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.066857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.067128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.067745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.070010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.097371 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1308/2684] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:05 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:06.101497 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.112068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e84935-3728-4396-9b20-4de889bdbc2a None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.169496 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.170105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.170423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.170652 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.171276 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.173281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.196419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1377/2685] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:06 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:06.200402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.214822 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fbcbe-3f9c-467f-884b-5c6d8381361a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.219521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.220170 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.220594 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.220878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.221509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.223115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.233924 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1309/2686] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:06.238469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.249163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a52f13-ac87-4704-b90a-6b821d4aef90 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.254344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.255063 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.255406 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.255742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] PATH_INFO: `/v3/projects/cab935164e1240d68a311863887a2a33/users/2b1830c50dd8455eac1645962d7d7d10/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.273015 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab935164e1240d68a311863887a2a33, user_id=2b1830c50dd8455eac1645962d7d7d10, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.275678 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.288554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1378/2687] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:06 2022] PUT /identity/v3/projects/cab935164e1240d68a311863887a2a33/users/2b1830c50dd8455eac1645962d7d7d10/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:06.292532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.302920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce76c0-5736-4c45-a4a7-29953ad0f854 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.361796 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.362550 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.362979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.363112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.363738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.365360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3453ee2d-2cf3-434a-b7f5-df342d3f26e9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.374965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1310/2688] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:06.379860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.390380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec48455-b3ac-43da-a9a5-6c66db08acc7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.395867 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.396517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.396797 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.397068 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] PATH_INFO: `/v3/projects/cab935164e1240d68a311863887a2a33/users/2b1830c50dd8455eac1645962d7d7d10/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.398980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab935164e1240d68a311863887a2a33, user_id=2b1830c50dd8455eac1645962d7d7d10, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.401665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.412238 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1379/2689] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:06 2022] PUT /identity/v3/projects/cab935164e1240d68a311863887a2a33/users/2b1830c50dd8455eac1645962d7d7d10/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:06.476824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.477112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.477390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.506125 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] MFA Rules not processed for user `2b1830c50dd8455eac1645962d7d7d10`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:06.507266 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:06.580227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1311/2690] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:06.591782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.593499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:06.603991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:06.652646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8dedce-3a82-48fb-a526-306b46b6b987 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.656306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.656906 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.657186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.657457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.666343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.667762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.677626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1380/2691] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:06.751916 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:06.762536 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58794d-7258-45f8-87fe-e27e85093ee1 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:06.766281 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:06.766877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:06.767154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:06.767424 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:06.773557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:06.827216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:06.828685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6126f-c906-4f18-8db3-a1215cbb3df2 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:06.837571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1312/2692] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:09.583339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.645511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96aa33f-0eef-4ed4-b06c-9101db2f8414 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.650043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.650669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.650948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.651221 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] PATH_INFO: `/v3/users/29e62588df6649fcb2b8b382ad638f52` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.652380 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29e62588df6649fcb2b8b382ad638f52)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.654308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.704438 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29e62588df6649fcb2b8b382ad638f52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:09.705098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29e62588df6649fcb2b8b382ad638f52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:09.712434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29e62588df6649fcb2b8b382ad638f52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:09.725095 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29e62588df6649fcb2b8b382ad638f52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:09.731921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29e62588df6649fcb2b8b382ad638f52'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:09.738286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1381/2693] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:09 2022] DELETE /identity/v3/users/29e62588df6649fcb2b8b382ad638f52 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:09.742769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.743240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.743338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.762935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:09.764196 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.822542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1313/2694] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:09.832477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:09.834010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:09.866128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:09.892855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d6bed-0b8d-4848-baf8-92abc3f06c78 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:09.896717 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:09.897358 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:09.897634 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:09.897905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:09.906729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:09.908189 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:09.917894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1382/2695] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:10.196833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.198279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.211142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c75a8c-2715-460c-900d-5a9802610cd4 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.216041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.216874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.217157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.217460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] PATH_INFO: `/v3/projects/c8364f16d5d34028be237c349e2bb88c` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.218584 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8364f16d5d34028be237c349e2bb88c)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.220531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.266418 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8364f16d5d34028be237c349e2bb88c', 'request_id': 'req-e46cd890-873d-4826-8627-23d3357698fd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.275345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8364f16d5d34028be237c349e2bb88c', 'request_id': 'req-e46cd890-873d-4826-8627-23d3357698fd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.281701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8364f16d5d34028be237c349e2bb88c', 'request_id': 'req-e46cd890-873d-4826-8627-23d3357698fd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.292985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] The token cache is being invalidate because project c8364f16d5d34028be237c349e2bb88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:10.293273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.293914 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.295390 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1314/2696] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:10 2022] DELETE /identity/v3/projects/c8364f16d5d34028be237c349e2bb88c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:10.299338 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.308841 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67597fdc-79c1-4b53-86e9-46683c509e74 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.364287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.364907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.365186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.365457 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] PATH_INFO: `/v3/users/2b1830c50dd8455eac1645962d7d7d10` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.366575 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1830c50dd8455eac1645962d7d7d10)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.368526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.417556 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1830c50dd8455eac1645962d7d7d10'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.418229 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1830c50dd8455eac1645962d7d7d10'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.425821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1830c50dd8455eac1645962d7d7d10'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.438226 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1830c50dd8455eac1645962d7d7d10'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.445333 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f613dfe0-ae69-40b7-a570-4a6158761e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1830c50dd8455eac1645962d7d7d10'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.451222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1383/2697] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:10 2022] DELETE /identity/v3/users/2b1830c50dd8455eac1645962d7d7d10 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:10.730342 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:10.731884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.746564 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46cd890-873d-4826-8627-23d3357698fd admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:10.775229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:10.804800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:10.806316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.806598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.806872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] PATH_INFO: `/v3/projects/cab935164e1240d68a311863887a2a33` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.808026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab935164e1240d68a311863887a2a33)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:10.809950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:10.857290 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab935164e1240d68a311863887a2a33', 'request_id': 'req-4839a6d9-3539-483f-bcc6-d4728ec220a0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.864698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab935164e1240d68a311863887a2a33', 'request_id': 'req-4839a6d9-3539-483f-bcc6-d4728ec220a0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.870795 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab935164e1240d68a311863887a2a33', 'request_id': 'req-4839a6d9-3539-483f-bcc6-d4728ec220a0'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.881021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] The token cache is being invalidate because project cab935164e1240d68a311863887a2a33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:10.881304 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.881935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-4839a6d9-3539-483f-bcc6-d4728ec220a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:10.883422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1315/2698] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:10 2022] DELETE /identity/v3/projects/cab935164e1240d68a311863887a2a33 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:10.889434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.890220 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.890322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:10.910632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:10.911852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:10.958030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:10.974948 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1384/2699] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:10.982907 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:10.983192 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:10.983465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.004411 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:11.005612 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.014480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1316/2700] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:11.020113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.030010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d36bce-bf74-4cd2-bcae-f601bc856833 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.035294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.035928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.036204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.036469 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.037112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:11.037408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.038991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.050542 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1385/2701] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:11.054925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.063935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877826b2-febe-4425-b462-e445fc1d0303 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.068581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.069171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.069447 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.069719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.070338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.072264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.101323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1317/2702] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:11.105529 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.115350 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2ebda-d624-423c-bd2a-4b99ac8fc715 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.170607 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.171199 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.171474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.171745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.172399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.174304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.195056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1386/2703] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:11.199204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.210176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a2212-4f65-4072-941c-c2e55e6c5323 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.214683 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.215275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.215548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.215875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.216509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.218083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.228338 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1318/2704] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:11.232128 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.242407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce9ceb-bf3e-4dac-b6f0-4f5e83b3bb9e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.248059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.248648 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.248928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.249196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.249825 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.251405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35868bf5-2564-4f19-af42-2f41612b2dc4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.262810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1387/2705] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:11.267145 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.278046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8898d-2bdd-47a1-86f1-886474a16bcb admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.282936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.283572 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.283882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.284153 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] PATH_INFO: `/v3/projects/222068bd59ef48c6a7ba98df190c10e5/users/44fb705fc59d4b26a84a8fd962765dc6/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.299716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=222068bd59ef48c6a7ba98df190c10e5, user_id=44fb705fc59d4b26a84a8fd962765dc6, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.303020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.320486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1319/2706] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:11 2022] PUT /identity/v3/projects/222068bd59ef48c6a7ba98df190c10e5/users/44fb705fc59d4b26a84a8fd962765dc6/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:11.330298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.330632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.330854 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.360201 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] MFA Rules not processed for user `44fb705fc59d4b26a84a8fd962765dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:11.361432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.427160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1388/2707] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:11.431702 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.490617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:11.495070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596160-8e2f-4a9e-b977-b33149c83133 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.501211 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.502013 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.502442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.502751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.504019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.505948 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.532108 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1320/2708] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:11.536285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.546984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c2d4-d530-488d-a30f-12d6738aea40 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.590689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:11.613181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.613799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.614077 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.614348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.614995 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.617141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.638977 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1389/2709] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:11.644469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.654955 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b687691-9ce0-42e7-a36b-f11b6d28262f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.659661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.660291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.660566 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.660832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.661465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.663043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.673408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1321/2710] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:11.677455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.687138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3f16f-6084-47e6-a3b6-04cb79beb6c7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.692293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.692886 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.693160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.693539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.694280 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.695898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461a82c-126b-47cc-87ad-033e7f233ed9 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.705958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1390/2711] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:11.710842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.720736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461dbd69-37f7-4dd1-8993-012d4b122522 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.725593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.726240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.726563 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.726837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] PATH_INFO: `/v3/projects/22acfa1813104f9a8b91fe591ccfa607/users/06d8fd59b3e64532a6ae340e65315595/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.741707 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22acfa1813104f9a8b91fe591ccfa607, user_id=06d8fd59b3e64532a6ae340e65315595, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.744521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.757465 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1322/2712] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:11 2022] PUT /identity/v3/projects/22acfa1813104f9a8b91fe591ccfa607/users/06d8fd59b3e64532a6ae340e65315595/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:11.769459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.769766 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.770041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.797716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] MFA Rules not processed for user `06d8fd59b3e64532a6ae340e65315595`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:11.798930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:11.864687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1391/2713] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:11.869327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.930250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee850b-eb9a-4751-bf60-6f9ccae4ee70 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:11.934809 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:11.935404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:11.935679 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:11.935983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:11.936596 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:11.938466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:11.964396 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1323/2714] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:11.968523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:11.979932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26af96-fb98-4b43-b7a5-df3296ae4197 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.035823 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.036429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.036712 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.036987 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.037632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.039561 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.061000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1392/2715] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:11 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:12.064954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.075934 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f03a8-93a2-4423-a800-e0cf2ec2423e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.080658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.081299 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.081716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.081994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.082659 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.084257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.094832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1324/2716] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:12.099650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.109585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d6de2-b6b2-4c08-9241-30af25f2a7f2 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.115281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.116018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.116304 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.116577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] PATH_INFO: `/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d/users/9c9696d5ab2a4467acc477279d108014/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.131570 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0fe0d64c854e9c8cef9ced4705395d, user_id=9c9696d5ab2a4467acc477279d108014, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.134266 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.147091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1393/2717] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:12 2022] PUT /identity/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d/users/9c9696d5ab2a4467acc477279d108014/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:12.151064 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.161968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4346053-0f5f-45dd-9a40-3e8061337c27 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.217878 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.218473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.218749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.219019 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.219640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.221427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fbc12-257b-4a4c-b03b-8686efbec55a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.232152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1325/2718] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:12.236253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.247138 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75158-8734-422a-b5da-22fd75353417 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.251714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.252385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.252664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.252937 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] PATH_INFO: `/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d/users/9c9696d5ab2a4467acc477279d108014/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.254848 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec0fe0d64c854e9c8cef9ced4705395d, user_id=9c9696d5ab2a4467acc477279d108014, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.257511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.267694 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1394/2719] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:12 2022] PUT /identity/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d/users/9c9696d5ab2a4467acc477279d108014/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:12.278269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.278558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.278832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.306793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] MFA Rules not processed for user `9c9696d5ab2a4467acc477279d108014`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:12.308045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.312994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:12.375436 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1326/2720] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:12.386752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:12.388480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:12.398855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:12.448571 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7f3013-0c10-41c1-bb2d-d9e35f0259ce admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:12.452470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:12.452803 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:12.454119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:12.454729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:12.525563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:12.526999 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:12.537176 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1395/2721] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:14.030122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:14.031772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:14.044834 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68053bef-bebf-4d0c-bcf5-d767de6104a7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:14.048337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:14.048939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:14.049216 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:14.049487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:14.105694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:14.107055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:14.116258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1327/2722] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:14.641041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:14.650539 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1251cad1-9b51-44e9-8868-2e8de52d64a7 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:14.654439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:14.655025 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:14.655308 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:14.655573 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:14.665596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:14.667006 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a1aeb7-8afe-49a2-b0ed-99585690d327 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:14.677487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1396/2723] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:18.537757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:18.599314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb0f04-6d68-4c0e-91f1-0a4da4c88184 service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:18.603812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:18.604438 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.604716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.604988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] PATH_INFO: `/v3/users/44fb705fc59d4b26a84a8fd962765dc6` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.606180 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44fb705fc59d4b26a84a8fd962765dc6)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:18.608175 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:18.652199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:18.655343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44fb705fc59d4b26a84a8fd962765dc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.656320 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44fb705fc59d4b26a84a8fd962765dc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.668280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44fb705fc59d4b26a84a8fd962765dc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.676156 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44fb705fc59d4b26a84a8fd962765dc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.683087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44fb705fc59d4b26a84a8fd962765dc6'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:18.688777 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1328/2724] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:18 2022] DELETE /identity/v3/users/44fb705fc59d4b26a84a8fd962765dc6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:18.693888 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.694407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.694407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.713340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:18.714466 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:18.773290 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1397/2725] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:18.782850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:18.784257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:18.839551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e11112-d19d-4062-856a-8dd83fa74fc8 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:18.843186 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:18.843779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:18.844085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:18.844352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:18.853738 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:18.855162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:18.865042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1329/2726] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:19.146970 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.148563 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:19.161293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb1c85-1f29-46e0-b6a3-9614f4018ff4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.166194 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.166975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.167316 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.167662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] PATH_INFO: `/v3/projects/222068bd59ef48c6a7ba98df190c10e5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.168932 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=222068bd59ef48c6a7ba98df190c10e5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.170908 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.216386 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:19.220507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '222068bd59ef48c6a7ba98df190c10e5', 'request_id': 'req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.228326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '222068bd59ef48c6a7ba98df190c10e5', 'request_id': 'req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.235233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '222068bd59ef48c6a7ba98df190c10e5', 'request_id': 'req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.246927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] The token cache is being invalidate because project 222068bd59ef48c6a7ba98df190c10e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:19.247217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.247871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.249349 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1398/2727] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:19 2022] DELETE /identity/v3/projects/222068bd59ef48c6a7ba98df190c10e5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:19.253366 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.263522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e451d-a8aa-4269-aeda-14103883ef6e service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.321935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.322574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.322868 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.323122 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] PATH_INFO: `/v3/users/06d8fd59b3e64532a6ae340e65315595` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.324283 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06d8fd59b3e64532a6ae340e65315595)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.326358 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.374991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06d8fd59b3e64532a6ae340e65315595'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.375650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06d8fd59b3e64532a6ae340e65315595'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.388940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06d8fd59b3e64532a6ae340e65315595'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.391204 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:19.399562 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06d8fd59b3e64532a6ae340e65315595'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.406929 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06d8fd59b3e64532a6ae340e65315595'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.413183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1330/2728] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:19 2022] DELETE /identity/v3/users/06d8fd59b3e64532a6ae340e65315595 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:19.686387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.687935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:19.703112 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea332a24-b1f4-41bb-bd6f-2aef2a48a551 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.762384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.762934 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.763215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.763495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] PATH_INFO: `/v3/projects/22acfa1813104f9a8b91fe591ccfa607` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.764681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22acfa1813104f9a8b91fe591ccfa607)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.766918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.813369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22acfa1813104f9a8b91fe591ccfa607', 'request_id': 'req-1b237538-1365-42d6-b306-af63f907a004'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.820668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22acfa1813104f9a8b91fe591ccfa607', 'request_id': 'req-1b237538-1365-42d6-b306-af63f907a004'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.827014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22acfa1813104f9a8b91fe591ccfa607', 'request_id': 'req-1b237538-1365-42d6-b306-af63f907a004'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.837443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] The token cache is being invalidate because project 22acfa1813104f9a8b91fe591ccfa607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:19.837727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.838372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.839872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1399/2729] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:19 2022] DELETE /identity/v3/projects/22acfa1813104f9a8b91fe591ccfa607 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:19.843725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:19.845165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:19.858310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acdd4e-d169-4d7b-aefb-70e3dd664083 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:19.913706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:19.914328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:19.914606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:19.914874 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] PATH_INFO: `/v3/users/9c9696d5ab2a4467acc477279d108014` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:19.915990 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9696d5ab2a4467acc477279d108014)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:19.917895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:19.962238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9696d5ab2a4467acc477279d108014'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.962872 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9696d5ab2a4467acc477279d108014'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.975538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9696d5ab2a4467acc477279d108014'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.982688 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9696d5ab2a4467acc477279d108014'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.989521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-26f922f6-bea4-4f35-9b39-e81a68567c10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9696d5ab2a4467acc477279d108014'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:19.995751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1331/2730] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:19 2022] DELETE /identity/v3/users/9c9696d5ab2a4467acc477279d108014 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:20.265907 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.267274 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.284065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b237538-1365-42d6-b306-af63f907a004 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.341240 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.341872 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.342151 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.342420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] PATH_INFO: `/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.343530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec0fe0d64c854e9c8cef9ced4705395d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.345477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.347876 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:20.395651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec0fe0d64c854e9c8cef9ced4705395d', 'request_id': 'req-cfacf300-6a82-41c6-acd9-5cf40eb16cac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.402914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec0fe0d64c854e9c8cef9ced4705395d', 'request_id': 'req-cfacf300-6a82-41c6-acd9-5cf40eb16cac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.409244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec0fe0d64c854e9c8cef9ced4705395d', 'request_id': 'req-cfacf300-6a82-41c6-acd9-5cf40eb16cac'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.419889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] The token cache is being invalidate because project ec0fe0d64c854e9c8cef9ced4705395d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:20.420184 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.420827 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-cfacf300-6a82-41c6-acd9-5cf40eb16cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:20.422300 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1400/2731] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:20 2022] DELETE /identity/v3/projects/ec0fe0d64c854e9c8cef9ced4705395d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:20.427152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.427441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.427717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.447372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.448623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.509838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1332/2732] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:20.520589 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.520817 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.521092 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.540974 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.542191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.550142 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1401/2733] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:20.554376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.559376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:20.565820 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e372819-9ac6-42b4-9d34-009a61dca344 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.572158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.572757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.573037 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.573313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.573954 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:20.574255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.575861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.588077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1333/2734] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:20.592591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.601968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9e67e-625a-471d-ad4b-311d2dd67fa4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.606786 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.607410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.607686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.607997 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.608622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.610538 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.639644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1402/2735] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:20.644088 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.654843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67910a-5327-4801-8c8b-9aaf8e8b1be9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.710840 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.711437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.711715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.712006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.712617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.714507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.734558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1334/2736] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:20.739852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.750118 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305090aa-bac4-4906-87eb-bf19b295f5a4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.755840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.756442 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.756723 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.756996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.757659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.759313 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.769637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1403/2737] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:20.773810 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.784223 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a30688-0099-451e-b609-024bdd41ff75 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.789131 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.789718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.789993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.790259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.790888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.792499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6d0dac-a727-4551-a284-24288e033a0f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.804565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1335/2738] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:20.808628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.819317 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fd193-72bc-4a66-ad82-fc0374eac3c9 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:20.824735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:20.825390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.825669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.825943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] PATH_INFO: `/v3/projects/4e123f9e41984e0383f6a94d58f490c5/users/ff00c288c1764b1688767e04b028ca76/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.840657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e123f9e41984e0383f6a94d58f490c5, user_id=ff00c288c1764b1688767e04b028ca76, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:20.843312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:20.856578 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1404/2739] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:20 2022] PUT /identity/v3/projects/4e123f9e41984e0383f6a94d58f490c5/users/ff00c288c1764b1688767e04b028ca76/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:17:20.867250 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:20.867753 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:20.867888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:20.895289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] MFA Rules not processed for user `ff00c288c1764b1688767e04b028ca76`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:20.896533 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:20.958505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1336/2740] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:20.968910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:20.970673 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:21.030348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23ab088-0ba5-42c2-af16-9933833e8703 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:21.033929 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:21.034532 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:21.034808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:21.035132 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:21.043339 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:21.044782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a61eb98-c539-4590-9240-691f5795b666 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:21.053960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1405/2741] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:30.880374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:30.939345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:30.943729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44747f6d-2b69-4238-9da5-348144ddb314 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:30.949160 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:30.949824 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:30.950105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:30.950378 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] PATH_INFO: `/v3/users/ff00c288c1764b1688767e04b028ca76` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:30.951496 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff00c288c1764b1688767e04b028ca76)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:30.953467 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.000556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff00c288c1764b1688767e04b028ca76'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.001215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff00c288c1764b1688767e04b028ca76'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.014062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff00c288c1764b1688767e04b028ca76'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.022189 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff00c288c1764b1688767e04b028ca76'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.029103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff00c288c1764b1688767e04b028ca76'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.034938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1337/2742] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:17:30 2022] DELETE /identity/v3/users/ff00c288c1764b1688767e04b028ca76 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:31.039685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.040259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.040368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.059525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:31.060795 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.091696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:31.127094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1406/2743] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:31.137480 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.138889 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.195246 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfd38a-630d-4b82-8428-3d29b94f5bf6 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.198812 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.199409 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.199684 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.199987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.209794 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.211255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ef7e6-1205-42f1-b945-f990e1887565 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.221693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1338/2744] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:31.486941 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.488851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.505861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812de6-84aa-4d26-950f-6fb1df71eaaa None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.510849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.511399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.511684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.511985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] PATH_INFO: `/v3/projects/4e123f9e41984e0383f6a94d58f490c5` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.513121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e123f9e41984e0383f6a94d58f490c5)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.515150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.564089 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e123f9e41984e0383f6a94d58f490c5', 'request_id': 'req-626f062a-2400-4155-9612-c4b840db6423'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.572467 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e123f9e41984e0383f6a94d58f490c5', 'request_id': 'req-626f062a-2400-4155-9612-c4b840db6423'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.578927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e123f9e41984e0383f6a94d58f490c5', 'request_id': 'req-626f062a-2400-4155-9612-c4b840db6423'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.590285 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] The token cache is being invalidate because project 4e123f9e41984e0383f6a94d58f490c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:17:31.590565 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.591196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-626f062a-2400-4155-9612-c4b840db6423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:17:31.592708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1407/2745] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:17:31 2022] DELETE /identity/v3/projects/4e123f9e41984e0383f6a94d58f490c5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:31.597522 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.597800 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.598258 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.617805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:31.618998 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.677162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1339/2746] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:31.685054 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.685345 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.685621 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.705698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:31.706923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:31.715828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1408/2747] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:31.719906 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.729425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ab319-25c4-4189-9602-c86e44ffa2eb None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.734188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.734789 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.735070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.735343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.736026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:17:31.736328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.737945 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.749647 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1340/2748] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:17:31.753777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.763685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e580eec-e26e-480c-8620-5af3a90c3d25 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.768829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.769430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.769738 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.770666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.771306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.773270 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.799524 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1409/2749] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:31 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:31.803701 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.813804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d29cc-bb91-497a-84f5-d8c6ccdf429c admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.818003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:31.872993 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.873599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.873883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.874150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.874765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.876841 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.898884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1341/2750] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:31 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:31.903036 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.914306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05079f5-ec16-4c88-9976-d3513ffc5bc0 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.918928 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.919508 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.919810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.920086 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.920725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.922314 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.932408 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1410/2751] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:31.936402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.947201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08ba0a-59f6-4aa3-bae2-6c670a571d61 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.951816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.952433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.952715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.952987 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:31.953625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:31.955213 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28999053-70e9-4747-85c0-42c8aea1d6cd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:31.966569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1342/2752] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:31.970595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:31.981504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445b913-df69-4ab3-b04c-13f566afc74c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:31.986218 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:31.986865 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:31.987141 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:31.987413 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] PATH_INFO: `/v3/projects/34855b10c042463d8c5067cef47b956d/users/2e137db209634ae4b760eba456203e66/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.003117 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34855b10c042463d8c5067cef47b956d, user_id=2e137db209634ae4b760eba456203e66, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:32.006868 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:32.019344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1411/2753] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:31 2022] PUT /identity/v3/projects/34855b10c042463d8c5067cef47b956d/users/2e137db209634ae4b760eba456203e66/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:32.024710 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31192135-e209-4687-bb8c-f205111f8123 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:32.024995 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31192135-e209-4687-bb8c-f205111f8123 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:32.025270 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31192135-e209-4687-bb8c-f205111f8123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.043686 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-31192135-e209-4687-bb8c-f205111f8123 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:32.044958 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-31192135-e209-4687-bb8c-f205111f8123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:32.103099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1343/2754] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:17:32.112662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:32.115465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:32.172475 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687735d6-e61f-4db4-8630-3862cb098228 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:32.176700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:32.177377 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:32.177651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:32.177921 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:32.184278 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:32.188894 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:32.190654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:32.201474 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1412/2755] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:36.123202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.123714 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.123849 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.154043 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] MFA Rules not processed for user `2e137db209634ae4b760eba456203e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:36.155183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:36.228523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1344/2756] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:36.232862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.242943 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d4b3-e75b-4131-be03-2e355b4c1af9 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.249796 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.250663 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.251157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.251635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.252580 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.255891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.285951 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1413/2757] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:17:36.290148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.303272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7afe51a-5ebf-4f55-ada4-f6df40b1f8e7 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.319150 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:36.398259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.399726 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.399981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.400469 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.400902 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.402974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.429600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1345/2758] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:17:36 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:17:36.430809 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.441696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3191c-325e-4559-877d-b1a7ab6db69b admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.450576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.451272 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.451551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.451853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.452504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.454119 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.469473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1414/2759] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:17:36.474259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.486838 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721dc2d-86c8-45e3-ab77-5a11fb519806 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.492231 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.493017 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.493368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.493715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] PATH_INFO: `/v3/projects/3d8ac6f22ebf448681daf161ace21b65/users/7caf67dcec664fe8be210ead7e51f54b/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.512339 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8ac6f22ebf448681daf161ace21b65, user_id=7caf67dcec664fe8be210ead7e51f54b, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.515318 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.531026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1346/2760] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:36 2022] PUT /identity/v3/projects/3d8ac6f22ebf448681daf161ace21b65/users/7caf67dcec664fe8be210ead7e51f54b/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:36.534912 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.546666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19768a17-455b-48b4-a3f2-2636e17a3266 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.612022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.612635 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.612910 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.613181 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.613821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.615444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef47093-1d0d-4a43-9f24-ed2acb5ab562 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.626170 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1415/2761] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:17:36.630646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:36.642323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4f807e-ebb5-4ce9-8dde-59934f3a06ae admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:36.648557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:36.649232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:36.649508 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:36.649778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] PATH_INFO: `/v3/projects/3d8ac6f22ebf448681daf161ace21b65/users/7caf67dcec664fe8be210ead7e51f54b/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:36.651703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d8ac6f22ebf448681daf161ace21b65, user_id=7caf67dcec664fe8be210ead7e51f54b, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:36.654416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:36.666450 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1347/2762] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:17:36 2022] PUT /identity/v3/projects/3d8ac6f22ebf448681daf161ace21b65/users/7caf67dcec664fe8be210ead7e51f54b/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:17:41.017743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3c0a-8bdc-43bb-a3b3-cd185429f95d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.018215 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3c0a-8bdc-43bb-a3b3-cd185429f95d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.018470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de3c0a-8bdc-43bb-a3b3-cd185429f95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.059162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f6de3c0a-8bdc-43bb-a3b3-cd185429f95d None None] MFA Rules not processed for user `7caf67dcec664fe8be210ead7e51f54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:17:41.061551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f6de3c0a-8bdc-43bb-a3b3-cd185429f95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.173108 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1416/2763] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:17:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:17:41.186327 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:17:41.189260 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:17:41.295355 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830cad7-5ec4-4ddd-ba7a-fa9a25a5cbb2 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:17:41.300949 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:17:41.301607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:17:41.301891 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:17:41.302159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:17:41.365898 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:17:41.367567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:17:41.377302 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1348/2764] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:17:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:17:51.165860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:17:51.373374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:06.283629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:06.284223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:06.284223 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:06.315725 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] MFA Rules not processed for user `2e137db209634ae4b760eba456203e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:06.317046 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:06.340768 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1417/2765] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:09.790862 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:09.792785 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:09.886567 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20fbe36-c3c9-4e0f-a2ce-d662a03a1fee service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:09.892353 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:09.892966 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:09.893254 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:09.893530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:09.895607 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:09.908940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:09.910534 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:09.921323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1349/2766] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:16.338735 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=15 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:19.916921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=36 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:20.957404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:20.971874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765d61ae-2c1c-44b8-8528-cd6d569abd67 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:20.977273 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:20.978222 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:20.978697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:20.978973 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:20.980776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:20.993980 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:20.996159 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:21.008204 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1418/2767] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:23.776544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:23.777962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:23.836230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38247a74-602c-43c3-abd1-7e5fd7682e6e service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:23.841112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:23.841837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:23.842159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:23.842503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:23.853832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:23.855431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:23.865259 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1350/2768] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:25.486623 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:25.511583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431844c-6d73-4a0f-b7ec-12c9109b175d service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:25.518018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:25.518891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:25.519369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:25.519873 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:25.524923 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:25.541183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:25.543402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:25.555378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1419/2769] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:33.860865 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=26 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:35.550586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=12 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:44.223468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:44.232961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93025a4-ae37-4c52-bc45-c80a80316d60 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:44.236585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:44.237176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:44.237453 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:44.237722 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:44.239435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:44.322763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:44.324748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:44.343556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1351/2770] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:47.146724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.149312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.212293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.214440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.227319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b3fb2-0549-479a-859c-55aa5a7c71f7 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.232875 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.233523 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.233804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.234079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] PATH_INFO: `/v3/users/1eb93e96417d4495b3577782be50bcbe` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.235225 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb93e96417d4495b3577782be50bcbe)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.237425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.282753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2617d2-8e49-45f1-a701-853bb1e70fe6 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.289144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.289901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.290173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.290444 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.292794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.304515 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb93e96417d4495b3577782be50bcbe'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.304945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.305441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb93e96417d4495b3577782be50bcbe'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.306572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.317216 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1420/2771] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:47.319100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb93e96417d4495b3577782be50bcbe'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.327444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb93e96417d4495b3577782be50bcbe'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.335440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-ea0e1818-e347-42ea-aca2-9f29d85e5442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb93e96417d4495b3577782be50bcbe'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.341994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1352/2772] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:47 2022] DELETE /identity/v3/users/1eb93e96417d4495b3577782be50bcbe => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:47.517417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.530754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eda41a-4000-4389-8087-30c779e4b90f service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.589382 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.590010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.590289 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.590560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] PATH_INFO: `/v3/projects/f82591f81f494cb2b76ead9e406e5b56` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.591706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82591f81f494cb2b76ead9e406e5b56)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.593640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.639335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82591f81f494cb2b76ead9e406e5b56', 'request_id': 'req-481fd861-1829-4af2-9688-15af4c4d776e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.646743 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82591f81f494cb2b76ead9e406e5b56', 'request_id': 'req-481fd861-1829-4af2-9688-15af4c4d776e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.652895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82591f81f494cb2b76ead9e406e5b56', 'request_id': 'req-481fd861-1829-4af2-9688-15af4c4d776e'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.659991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:47.664775 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] The token cache is being invalidate because project f82591f81f494cb2b76ead9e406e5b56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:47.665361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.666792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-481fd861-1829-4af2-9688-15af4c4d776e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:47.668516 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1421/2773] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:47 2022] DELETE /identity/v3/projects/f82591f81f494cb2b76ead9e406e5b56 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:47.673921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.674232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.674505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.703483 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:47.704631 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.727888 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:47.763182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1353/2774] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:47.771810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.772098 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.772366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.792080 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:47.793487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:47.801244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1422/2775] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:47.805376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.815965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c481088f-cbf2-4b1b-b63f-3ed7806fda61 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.820505 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.821090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.821362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.821626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.822253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:47.822617 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.824538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.835755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1354/2776] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:47.839916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.849005 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821d275-f9ff-41d7-9d9c-0fefc814cfa5 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.853579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.854769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.855043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.855307 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.855984 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.857880 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.884303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1423/2777] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:47.888779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.899255 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1b99f-eb0d-40bd-bff2-de0be801f46e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.954279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.954827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:47.955104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:47.955373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:47.956011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:47.957908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:47.978343 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1355/2778] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:47 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:47.983198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:47.993477 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cb305-27eb-4609-a082-eb90c04aaaca admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:47.999217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:47.999878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.000161 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.000432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.001067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.002665 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.012905 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1424/2779] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:48.016855 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.027666 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330d8e1-0ebd-4570-aea6-ffb5a3918fa0 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.032256 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.032843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.033123 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.033391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.034021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.035685 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8e1e00-ad1a-4b6b-8932-af224b4688fa admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.046836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1356/2780] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:48.051113 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.061697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b51068-4b51-4291-8612-5001585f868c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.067360 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.068511 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.069027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.069318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] PATH_INFO: `/v3/projects/3b748d389e7d477dbdd43b0dab40ba50/users/426d5612a148464d9623a8b767d9ba7e/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.084637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b748d389e7d477dbdd43b0dab40ba50, user_id=426d5612a148464d9623a8b767d9ba7e, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.087303 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.101369 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1425/2781] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:48 2022] PUT /identity/v3/projects/3b748d389e7d477dbdd43b0dab40ba50/users/426d5612a148464d9623a8b767d9ba7e/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:48.112600 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.112884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.113237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.142053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] MFA Rules not processed for user `426d5612a148464d9623a8b767d9ba7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:48.143252 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.208576 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1357/2782] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:48.220693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:48.222792 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:48.303133 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019ba65-5b7b-49b9-b96c-6d296489101c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:48.309569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:48.310183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:48.310462 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:48.310734 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:48.318830 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:48.321019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315dbf1-7345-42d6-b215-7d07dd159c30 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:48.331993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1426/2783] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:57.699079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:57.724938 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:57.765197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bc48f-9850-4331-8f86-4f261acc6d3a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:57.770178 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:57.770751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:57.771030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:57.771325 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] PATH_INFO: `/v3/users/426d5612a148464d9623a8b767d9ba7e` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:57.772468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=426d5612a148464d9623a8b767d9ba7e)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:57.775197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:57.821828 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426d5612a148464d9623a8b767d9ba7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:57.822487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426d5612a148464d9623a8b767d9ba7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:57.834115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426d5612a148464d9623a8b767d9ba7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:57.841351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426d5612a148464d9623a8b767d9ba7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:57.848402 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426d5612a148464d9623a8b767d9ba7e'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:57.854398 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1358/2784] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:18:57 2022] DELETE /identity/v3/users/426d5612a148464d9623a8b767d9ba7e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:57.858855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:57.859355 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:57.859459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:57.878533 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:57.879812 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:57.940024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1427/2785] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:57.950759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:57.952193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.014763 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384e8504-bd9b-47cd-80d8-71d5e11a4e3a admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.018413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.019280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.019555 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.019881 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.029431 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.030788 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9eb29-975b-4fd7-9796-c247f9057ec0 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.040986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1359/2786] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:58.305588 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.307014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.320745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96655b-0da5-4895-a5fb-c2f84119b469 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.325441 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.326029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.326330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.326595 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] PATH_INFO: `/v3/projects/3b748d389e7d477dbdd43b0dab40ba50` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.328342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b748d389e7d477dbdd43b0dab40ba50)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.330465 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.348148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:58.378640 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b748d389e7d477dbdd43b0dab40ba50', 'request_id': 'req-a1314b59-9a30-4d85-955f-b74d1a2f217a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:58.385835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b748d389e7d477dbdd43b0dab40ba50', 'request_id': 'req-a1314b59-9a30-4d85-955f-b74d1a2f217a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:58.392114 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b748d389e7d477dbdd43b0dab40ba50', 'request_id': 'req-a1314b59-9a30-4d85-955f-b74d1a2f217a'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:58.402757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] The token cache is being invalidate because project 3b748d389e7d477dbdd43b0dab40ba50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:18:58.403047 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:58.403683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a1314b59-9a30-4d85-955f-b74d1a2f217a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:18:58.405191 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1428/2787] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:18:58 2022] DELETE /identity/v3/projects/3b748d389e7d477dbdd43b0dab40ba50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:18:58.410070 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.410352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.410623 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.436421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:58.437644 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.480502 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:58.500020 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1360/2788] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:18:58.507988 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.508432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.508706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.528674 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:58.529871 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.537612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1429/2789] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:58.542341 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.552111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914fb63-ca43-4d47-90c2-856e6828bb89 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.556781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.557372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.557651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.557920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.558561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:18:58.558861 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.560487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.572045 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1361/2790] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:18:58.576186 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.585246 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836e973-7959-43b9-be62-ba8bcbf5f9cd None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.590855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.591439 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.591714 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.592017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.592628 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.594509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.622231 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1430/2791] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:58 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:58.626330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.636588 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5559-37ea-440f-ac69-d023180265d3 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.692097 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.692696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.692971 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.693237 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.693845 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.696893 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.721228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1362/2792] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:58 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:58.725407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.736451 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa29a686-5cbf-4f69-9023-65382114412a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.741777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.742596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.749340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.749340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.749340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.749340 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.756515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1431/2793] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:58.760804 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.771334 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da9967-64ca-4be5-bef7-852d412da0e9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.776524 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.777120 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.777405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.777681 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.778323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.779974 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e839eb-0315-48b2-b8ef-1a0a633c3aac admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.791689 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1363/2794] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:58.795840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.805914 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab0b71-affc-44de-b8bc-cbd22154fac1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:58.810693 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:58.811334 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.811611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.811918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] PATH_INFO: `/v3/projects/64647a6bc83045a6914d2316510894d6/users/eebc59a6100a4888b84854b74662786b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.826173 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64647a6bc83045a6914d2316510894d6, user_id=eebc59a6100a4888b84854b74662786b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:58.828889 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:58.843599 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1432/2795] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:58 2022] PUT /identity/v3/projects/64647a6bc83045a6914d2316510894d6/users/eebc59a6100a4888b84854b74662786b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:58.848625 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb775-e82f-41a0-9ed1-51ebb0a2595d None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:58.848956 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb775-e82f-41a0-9ed1-51ebb0a2595d None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:58.849229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71fb775-e82f-41a0-9ed1-51ebb0a2595d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:58.868460 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c71fb775-e82f-41a0-9ed1-51ebb0a2595d None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:58.869669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c71fb775-e82f-41a0-9ed1-51ebb0a2595d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.928699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1364/2796] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:58.938612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:58.940427 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:58.996748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c025d1-4492-456b-87b6-eb6bd5540212 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:59.000776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:59.001417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.001746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.002020 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.011716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:59.013164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:59.022763 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1433/2797] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:18:59.640069 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.640351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.640626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.671491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] MFA Rules not processed for user `eebc59a6100a4888b84854b74662786b`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:18:59.672729 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:18:59.721983 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:18:59.747739 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1365/2798] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:18:59 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:18:59.752298 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:59.762762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7073-8e3d-49cc-acaf-0dc5e4fbfd3a service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:59.767437 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:59.768202 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.768754 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.769057 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.769685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:59.771627 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:59.799344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1434/2799] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:18:59 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:59.803630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:59.815782 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe397c-5717-47fc-98ae-085306bf9869 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:59.877920 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:59.878545 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.878852 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.879127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.879743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:59.881718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:59.903778 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1366/2800] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:18:59 2022] POST /identity/v3/users => generated 309 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:18:59.907972 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:59.919065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ada0a86-39cf-49b7-9d73-8f34084ded09 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:59.923505 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:59.924146 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.924430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.924700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.925332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:59.926913 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:59.938206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1435/2801] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:18:59.942407 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:18:59.952475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb676cd-e5cf-4baf-ada8-050b3b8dcb13 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:18:59.957050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:18:59.957690 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:18:59.957968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:18:59.958238 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] PATH_INFO: `/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6/users/c777e60d0d00486a84b65379a7ee6a17/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:18:59.972895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbeac4dc20484d6e8e3348846aeb31a6, user_id=c777e60d0d00486a84b65379a7ee6a17, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:18:59.975544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:18:59.988761 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1367/2802] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:18:59 2022] PUT /identity/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6/users/c777e60d0d00486a84b65379a7ee6a17/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:18:59.993428 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:00.004368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d794f-03b7-4c81-b6e1-cbf68958deff admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:00.012331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:00.063677 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:00.064312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:00.064591 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:00.064864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:00.065503 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:00.067094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a2ed0-9ac1-4364-816f-df5e7366e2b1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:00.077435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1436/2803] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:19:00.081574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:00.092077 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5caf-71f3-4c4e-9ab5-e5bf69d3dd83 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:00.096839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:00.097489 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:00.097765 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:00.098032 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] PATH_INFO: `/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6/users/c777e60d0d00486a84b65379a7ee6a17/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:00.100115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbeac4dc20484d6e8e3348846aeb31a6, user_id=c777e60d0d00486a84b65379a7ee6a17, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:00.102950 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:00.113202 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1368/2804] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:00 2022] PUT /identity/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6/users/c777e60d0d00486a84b65379a7ee6a17/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:00.713651 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:00.713938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:00.714213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:00.743234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] MFA Rules not processed for user `c777e60d0d00486a84b65379a7ee6a17`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:00.744487 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:00.818318 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1437/2805] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:02.097579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:02.160581 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f464e-4ba5-4edd-8fec-ceb6e0d68e31 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:02.165277 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:02.165903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:02.166184 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:02.166455 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] PATH_INFO: `/v3/users/eebc59a6100a4888b84854b74662786b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:02.167590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eebc59a6100a4888b84854b74662786b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:02.169558 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:02.208630 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:02.218615 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eebc59a6100a4888b84854b74662786b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.219499 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eebc59a6100a4888b84854b74662786b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.232350 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eebc59a6100a4888b84854b74662786b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.240463 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eebc59a6100a4888b84854b74662786b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.247395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eebc59a6100a4888b84854b74662786b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.253227 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1369/2806] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:02 2022] DELETE /identity/v3/users/eebc59a6100a4888b84854b74662786b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:02.430564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:02.442324 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b315621f-716c-4b23-a5d3-5ed0a8afe859 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:02.500978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:02.501594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:02.501877 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:02.502149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] PATH_INFO: `/v3/projects/64647a6bc83045a6914d2316510894d6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:02.503301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=64647a6bc83045a6914d2316510894d6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:02.505611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:02.544569 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:02.556049 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64647a6bc83045a6914d2316510894d6', 'request_id': 'req-9c17ef89-81ee-4db8-8cea-ee6e108336dc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.563387 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64647a6bc83045a6914d2316510894d6', 'request_id': 'req-9c17ef89-81ee-4db8-8cea-ee6e108336dc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.569824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64647a6bc83045a6914d2316510894d6', 'request_id': 'req-9c17ef89-81ee-4db8-8cea-ee6e108336dc'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.580264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] The token cache is being invalidate because project 64647a6bc83045a6914d2316510894d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:02.580548 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.581171 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.583039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1438/2807] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:02 2022] DELETE /identity/v3/projects/64647a6bc83045a6914d2316510894d6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:19:02.588026 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:02.589459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:02.603041 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5f6b1-864c-401e-aad6-837f036ac658 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:02.658979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:02.659613 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:02.659939 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:02.660240 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] PATH_INFO: `/v3/users/c777e60d0d00486a84b65379a7ee6a17` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:02.661374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c777e60d0d00486a84b65379a7ee6a17)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:02.663464 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:02.709058 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c777e60d0d00486a84b65379a7ee6a17'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.710860 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c777e60d0d00486a84b65379a7ee6a17'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.722947 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c777e60d0d00486a84b65379a7ee6a17'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.731417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c777e60d0d00486a84b65379a7ee6a17'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.739478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-a0a3bc12-3e31-48c8-8add-72ead036b1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c777e60d0d00486a84b65379a7ee6a17'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:02.749766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1370/2808] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:02 2022] DELETE /identity/v3/users/c777e60d0d00486a84b65379a7ee6a17 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:02.938682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:02.940164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:02.959105 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17ef89-81ee-4db8-8cea-ee6e108336dc admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.025862 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.026418 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.026695 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.026964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] PATH_INFO: `/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.028147 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbeac4dc20484d6e8e3348846aeb31a6)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.030102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.082584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbeac4dc20484d6e8e3348846aeb31a6', 'request_id': 'req-79b0245d-5fd0-41aa-bf96-af6b19e6134b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:03.091518 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbeac4dc20484d6e8e3348846aeb31a6', 'request_id': 'req-79b0245d-5fd0-41aa-bf96-af6b19e6134b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:03.101342 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbeac4dc20484d6e8e3348846aeb31a6', 'request_id': 'req-79b0245d-5fd0-41aa-bf96-af6b19e6134b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:03.114486 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] The token cache is being invalidate because project dbeac4dc20484d6e8e3348846aeb31a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:03.114824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:03.115499 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-79b0245d-5fd0-41aa-bf96-af6b19e6134b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:03.117683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1439/2809] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:02 2022] DELETE /identity/v3/projects/dbeac4dc20484d6e8e3348846aeb31a6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:03.123093 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.123590 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.123708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.145850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:03.147205 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:03.181901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:03.216364 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1371/2810] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:03.224196 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.224512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.225058 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.249901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:03.251154 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:03.259554 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1440/2811] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:03.263903 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.275425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073b63a-abbc-4fde-8b45-5127271254b3 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.280165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.280715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.281024 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.281297 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.281953 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:19:03.282253 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.283922 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.298882 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1372/2812] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:19:03.304239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.314826 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f1ec55-04f4-41fc-86cf-558e54d421e4 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.320139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.320749 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.321026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.321297 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.321920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.323858 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.354135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1441/2813] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:03.358419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.374140 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b914706-6c4d-4b58-a37a-0f1c7d517a85 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.446042 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.446837 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.447115 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.447413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.448086 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.450036 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.475720 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1373/2814] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:03.481644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.494657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342385e-ef98-470e-adcc-bcb53271c7a7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.499963 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.500878 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.501390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.501696 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.502374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.504550 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.516321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1442/2815] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:03.521275 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.532972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f8fa22-ec49-471e-a93c-294d91bce409 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.538507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.539245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.539525 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.539823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.540462 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.543408 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accbf58e-4d1b-4fcc-b6c0-246e92804ec5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.559532 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1374/2816] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:19:03.564283 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.575390 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355a66c-6544-44fb-85aa-f8e30ae275a6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.581065 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.581753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.582032 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.582306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] PATH_INFO: `/v3/projects/80b5c3f4d2ce4b259f537e9acfaa9154/users/49f61aa326da4e1883bc92e086188ed8/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.595060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:03.602708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b5c3f4d2ce4b259f537e9acfaa9154, user_id=49f61aa326da4e1883bc92e086188ed8, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.606124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.621820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1443/2817] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:03 2022] PUT /identity/v3/projects/80b5c3f4d2ce4b259f537e9acfaa9154/users/49f61aa326da4e1883bc92e086188ed8/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:03.633388 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.633699 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.634146 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.667859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] MFA Rules not processed for user `49f61aa326da4e1883bc92e086188ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:03.670102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:03.740060 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1375/2818] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:19:03.745148 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.807431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecbba4-9746-4534-a1d9-4ef83218586c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.812060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.812662 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.812938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.813206 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.813840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.815730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.848531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1444/2819] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:03.853262 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.863975 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9823ee-5e0a-4260-8086-ab4953659d7e None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.888798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:03.921543 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.922154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.922433 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.922706 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.923323 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.925282 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.945979 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1376/2820] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:19:03 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:03.950004 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.961212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909b8a3-14c0-4da3-8a82-e2a828fcbff7 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:03.965821 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:03.966417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:03.966701 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:03.966991 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:03.967616 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:03.969255 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:03.980802 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1445/2821] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:03.984908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:03.995293 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5d54aa-56d3-44d7-82dd-d8c628d3dbb5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.000432 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.001183 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.001507 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.001856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] PATH_INFO: `/v3/projects/93e65601e1a24aa182f6c24ad0b80290/users/097cfea72d6e4f1bb1c399adaba50786/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.016821 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e65601e1a24aa182f6c24ad0b80290, user_id=097cfea72d6e4f1bb1c399adaba50786, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.019877 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.034560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1377/2822] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:03 2022] PUT /identity/v3/projects/93e65601e1a24aa182f6c24ad0b80290/users/097cfea72d6e4f1bb1c399adaba50786/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:04.039502 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:04.049596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ec874-fd8b-487c-b7ab-2ce571dc2c05 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.106042 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.106560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.106840 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.107115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.107752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.109376 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0fd0d6-d706-4888-8046-bd253e3dda79 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.119008 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1446/2823] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:04.123386 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:04.134374 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89179670-c2cb-4255-9be3-6842b0819e6d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.138848 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.139493 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.139766 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.140062 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] PATH_INFO: `/v3/projects/93e65601e1a24aa182f6c24ad0b80290/users/097cfea72d6e4f1bb1c399adaba50786/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.141968 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e65601e1a24aa182f6c24ad0b80290, user_id=097cfea72d6e4f1bb1c399adaba50786, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.144884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.155629 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1378/2824] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:04 2022] PUT /identity/v3/projects/93e65601e1a24aa182f6c24ad0b80290/users/097cfea72d6e4f1bb1c399adaba50786/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:04.166019 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.166306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.166581 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.195700 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] MFA Rules not processed for user `097cfea72d6e4f1bb1c399adaba50786`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:04.196975 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:04.243955 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:04.269702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1447/2825] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:04 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:04.280863 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:04.282557 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:04.342050 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9b8176-bcb0-4c21-b83b-edc80a170a62 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.345759 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.346362 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.346639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.346912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.355991 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.357478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b7de7-90fe-40ec-adb3-bdcfa86d5a43 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.366746 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1379/2826] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:04.501924 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:04.564509 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7b084-5c91-4464-92d6-d14eeef87e49 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.569617 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.570233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.570517 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.570794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] PATH_INFO: `/v3/users/49f61aa326da4e1883bc92e086188ed8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.571953 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49f61aa326da4e1883bc92e086188ed8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.573884 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.619149 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49f61aa326da4e1883bc92e086188ed8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:04.619780 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49f61aa326da4e1883bc92e086188ed8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:04.627144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49f61aa326da4e1883bc92e086188ed8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:04.639672 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49f61aa326da4e1883bc92e086188ed8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:04.646601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49f61aa326da4e1883bc92e086188ed8'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:04.652545 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1448/2827] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:04 2022] DELETE /identity/v3/users/49f61aa326da4e1883bc92e086188ed8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:04.656961 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.657330 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.657610 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.676171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:04.677457 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:04.735126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1380/2828] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:04.745234 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:04.746637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:04.802661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93ae1c-f625-4ccf-b81a-b3d7c8bfc9f6 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:04.806936 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:04.807530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:04.807837 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:04.808115 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:04.816961 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:04.819219 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:04.828579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1449/2829] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:05.102313 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:05.103911 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:05.119781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb2e4b1-9643-4bc5-beb2-9403ebb90d59 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:05.125527 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:05.126201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.126658 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.127011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] PATH_INFO: `/v3/projects/80b5c3f4d2ce4b259f537e9acfaa9154` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.128491 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b5c3f4d2ce4b259f537e9acfaa9154)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:05.130718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:05.153212 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:05.181506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b5c3f4d2ce4b259f537e9acfaa9154', 'request_id': 'req-896431f9-ddce-4628-82c6-b0b0035e2994'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.190103 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b5c3f4d2ce4b259f537e9acfaa9154', 'request_id': 'req-896431f9-ddce-4628-82c6-b0b0035e2994'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.197420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b5c3f4d2ce4b259f537e9acfaa9154', 'request_id': 'req-896431f9-ddce-4628-82c6-b0b0035e2994'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.209704 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] The token cache is being invalidate because project 80b5c3f4d2ce4b259f537e9acfaa9154 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:05.210147 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.210912 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.215400 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1381/2830] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:05 2022] DELETE /identity/v3/projects/80b5c3f4d2ce4b259f537e9acfaa9154 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:05.216622 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:05.227481 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe1bad3-b6c3-4be8-9c81-562821a43872 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:05.231391 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:05.290703 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:05.291335 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.291611 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.291916 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] PATH_INFO: `/v3/users/097cfea72d6e4f1bb1c399adaba50786` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.293044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=097cfea72d6e4f1bb1c399adaba50786)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:05.294944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:05.344332 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '097cfea72d6e4f1bb1c399adaba50786'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.345022 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '097cfea72d6e4f1bb1c399adaba50786'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.353729 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '097cfea72d6e4f1bb1c399adaba50786'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.366417 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '097cfea72d6e4f1bb1c399adaba50786'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.373384 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-a3fcc2a4-0a7f-4a90-a469-85e67aff8aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '097cfea72d6e4f1bb1c399adaba50786'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.379686 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1450/2831] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:05 2022] DELETE /identity/v3/users/097cfea72d6e4f1bb1c399adaba50786 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:05.682517 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:05.685073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:05.702247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896431f9-ddce-4628-82c6-b0b0035e2994 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:05.764798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:05.764798 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.765055 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.765321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] PATH_INFO: `/v3/projects/93e65601e1a24aa182f6c24ad0b80290` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.766427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e65601e1a24aa182f6c24ad0b80290)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:05.768651 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:05.817118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e65601e1a24aa182f6c24ad0b80290', 'request_id': 'req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.826672 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e65601e1a24aa182f6c24ad0b80290', 'request_id': 'req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.833162 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e65601e1a24aa182f6c24ad0b80290', 'request_id': 'req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.844981 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] The token cache is being invalidate because project 93e65601e1a24aa182f6c24ad0b80290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:05.845273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.845926 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-243fb73b-50b9-4c6e-8fc0-c09ff706c0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:05.847421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1382/2832] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:05 2022] DELETE /identity/v3/projects/93e65601e1a24aa182f6c24ad0b80290 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:05.853041 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.853594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.853594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.872860 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:05.874078 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:05.934367 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1451/2833] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:05.942389 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.942781 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.943662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.964010 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:05.965247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:05.973185 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1383/2834] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:05.977944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:05.987654 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686ee023-8118-41a5-a5fe-0daec1d63f6f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:05.993211 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:05.993819 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:05.994102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:05.994378 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:05.995031 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:19:05.995337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:05.996978 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.009124 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1452/2835] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:19:06.013415 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.023511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531be453-7297-4e9d-9fad-1f3fe3a774b0 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.028171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.028769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.029039 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.029310 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.029972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.031969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.059542 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1384/2836] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:19:06.064993 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.075072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9a7aa-8fc0-42d6-89b9-a2896b5c8053 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.084385 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:06.131682 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.132409 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.132671 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.132945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.133568 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.135495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.156708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1453/2837] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:06.160806 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.172459 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69838886-9945-4fd6-a682-e13e4df5d75d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.176978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.177814 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.178264 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.178538 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.179172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.180780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.182769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:06.192395 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1385/2838] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:06.196680 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.206488 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d77f98-7457-4c92-a499-68514e6b2935 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.211629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.212253 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.212523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.212789 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.213448 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.215030 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722ca7b-011c-4fb0-96e8-9829e0a1b730 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.226244 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1454/2839] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:06.230357 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.240646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529f4faf-8501-440f-a997-c0435987aad4 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.245300 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.246011 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.246287 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.246553 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] PATH_INFO: `/v3/projects/5b1d79e29efb454892a4bf7beeebe783/users/927c5e67ff304aff9220d30f6002d411/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.262750 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b1d79e29efb454892a4bf7beeebe783, user_id=927c5e67ff304aff9220d30f6002d411, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.265488 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.279054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1386/2840] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:06 2022] PUT /identity/v3/projects/5b1d79e29efb454892a4bf7beeebe783/users/927c5e67ff304aff9220d30f6002d411/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:06.345612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.345960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.346131 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.373831 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] MFA Rules not processed for user `927c5e67ff304aff9220d30f6002d411`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:06.375026 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:06.437431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1455/2841] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:06.441999 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.503640 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563d77-9f3a-46db-bbea-85194305d472 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.508952 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.509556 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.509844 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.510118 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.510749 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.512717 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.541044 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1387/2842] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:19:06.545209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.556614 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed06e0-dee6-41d1-ab94-de21a1f3ff8d None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.613436 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.614035 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.614396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.614667 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.615844 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.617762 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.639079 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1456/2843] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:06.643419 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.654621 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a6900-35ea-4647-b882-74edaee9d6e5 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.659387 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.660053 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.660328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.660599 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.661314 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.663473 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.673925 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1388/2844] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:06.678643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.688429 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df687-240e-4f47-a9ee-8d657cb4768d admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.694099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.694748 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.695027 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.695301 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] PATH_INFO: `/v3/projects/7a53502e395a4c96879bcdfbbe79d727/users/473ac381e0b94276bd0e15119b387871/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.704719 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:06.713043 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a53502e395a4c96879bcdfbbe79d727, user_id=473ac381e0b94276bd0e15119b387871, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.715885 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.729053 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1457/2845] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:06 2022] PUT /identity/v3/projects/7a53502e395a4c96879bcdfbbe79d727/users/473ac381e0b94276bd0e15119b387871/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:06.733187 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.744157 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb07b0b-91e4-490b-b6c7-8c53679d5992 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.799461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.800099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.800382 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.800656 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.801295 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.803571 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff119cb-d037-4a4c-9b3b-a7b6c5cd73da admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.813768 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1389/2846] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:06.818900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.829259 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c9485-817a-4e3f-878f-8930950a51d3 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:06.833904 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:06.834551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.834828 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.835243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] PATH_INFO: `/v3/projects/7a53502e395a4c96879bcdfbbe79d727/users/473ac381e0b94276bd0e15119b387871/roles/d58e514376d0402cb9e57a1f44d5d41d` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.837208 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a53502e395a4c96879bcdfbbe79d727, user_id=473ac381e0b94276bd0e15119b387871, role_id=d58e514376d0402cb9e57a1f44d5d41d)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:06.839874 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:06.850102 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1458/2847] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:06 2022] PUT /identity/v3/projects/7a53502e395a4c96879bcdfbbe79d727/users/473ac381e0b94276bd0e15119b387871/roles/d58e514376d0402cb9e57a1f44d5d41d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:06.860201 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:06.860487 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:06.860762 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:06.889478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] MFA Rules not processed for user `473ac381e0b94276bd0e15119b387871`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:06.890680 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:06.921528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:06.958018 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1390/2848] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:06.970010 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:06.971747 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:07.029040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e84cba-21e0-41fe-9c96-cdbcab46203a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:07.032646 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:07.033237 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:07.033510 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:07.033776 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:07.089195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:07.090638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:07.099388 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1459/2849] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:11.298817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:11.300263 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:11.312751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4baf8ea-3071-4c98-bd13-fb3d47a2e35a None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:11.316271 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:11.316876 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:11.317695 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:11.317969 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:11.326751 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:11.328207 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a1d40e-d012-4704-9f8d-21a6b67edb71 service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:11.337569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1391/2850] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:16.221919 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:16.284379 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c3e8d6-900e-4769-bf08-b00aeff70749 service cinder] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:16.289985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:16.290602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:16.290883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:16.291153 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] PATH_INFO: `/v3/users/927c5e67ff304aff9220d30f6002d411` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:16.292320 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=927c5e67ff304aff9220d30f6002d411)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:16.294228 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:16.326044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:16.341626 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927c5e67ff304aff9220d30f6002d411'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.342330 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927c5e67ff304aff9220d30f6002d411'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.350526 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927c5e67ff304aff9220d30f6002d411'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.362311 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927c5e67ff304aff9220d30f6002d411'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.369293 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927c5e67ff304aff9220d30f6002d411'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.375059 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1460/2851] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:16 2022] DELETE /identity/v3/users/927c5e67ff304aff9220d30f6002d411 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:16.379486 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:16.380006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:16.380119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:16.408235 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:16.409466 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:16.471080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1392/2852] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:16.481399 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:16.482810 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:16.562947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a7672d-c1d0-4d5b-95f7-2603eb7d5a3e admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:16.567365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:16.568612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:16.568612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:16.568612 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:16.577644 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:16.579084 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:16.588584 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1461/2853] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:16.859654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:16.861280 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:16.874360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a2ba-a0f3-454c-b454-836c93cba245 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:16.879637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:16.880308 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:16.880586 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:16.880857 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] PATH_INFO: `/v3/projects/5b1d79e29efb454892a4bf7beeebe783` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:16.881978 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b1d79e29efb454892a4bf7beeebe783)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:16.884078 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:16.932337 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b1d79e29efb454892a4bf7beeebe783', 'request_id': 'req-25da2988-806c-465f-a4ff-626c206984cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.939372 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b1d79e29efb454892a4bf7beeebe783', 'request_id': 'req-25da2988-806c-465f-a4ff-626c206984cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.945833 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b1d79e29efb454892a4bf7beeebe783', 'request_id': 'req-25da2988-806c-465f-a4ff-626c206984cd'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.958560 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] The token cache is being invalidate because project 5b1d79e29efb454892a4bf7beeebe783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:16.958850 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.959506 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:16.961006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1393/2854] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:16 2022] DELETE /identity/v3/projects/5b1d79e29efb454892a4bf7beeebe783 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:16.964927 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:16.977267 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187b207-edb4-4172-8b64-8b103642fa30 service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.034239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.035730 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.036039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.036309 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] PATH_INFO: `/v3/users/473ac381e0b94276bd0e15119b387871` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.037422 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=473ac381e0b94276bd0e15119b387871)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.039328 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.073531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:17.087938 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '473ac381e0b94276bd0e15119b387871'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.088582 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '473ac381e0b94276bd0e15119b387871'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.095742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '473ac381e0b94276bd0e15119b387871'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.108759 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '473ac381e0b94276bd0e15119b387871'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.115602 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-09274030-bb27-4096-98fd-a41336a462ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '473ac381e0b94276bd0e15119b387871'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.121447 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1462/2855] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:19:16 2022] DELETE /identity/v3/users/473ac381e0b94276bd0e15119b387871 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:17.384901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.386420 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:17.403514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da2988-806c-465f-a4ff-626c206984cd admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.435650 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:17.466363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.467035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.467316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.467611 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] PATH_INFO: `/v3/projects/7a53502e395a4c96879bcdfbbe79d727` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.468823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a53502e395a4c96879bcdfbbe79d727)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.470780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.517442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a53502e395a4c96879bcdfbbe79d727', 'request_id': 'req-708121d2-7605-4af0-84f6-875ec920c065'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.524606 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a53502e395a4c96879bcdfbbe79d727', 'request_id': 'req-708121d2-7605-4af0-84f6-875ec920c065'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.530892 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a53502e395a4c96879bcdfbbe79d727', 'request_id': 'req-708121d2-7605-4af0-84f6-875ec920c065'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.541723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] The token cache is being invalidate because project 7a53502e395a4c96879bcdfbbe79d727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:19:17.542007 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.542639 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-708121d2-7605-4af0-84f6-875ec920c065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:19:17.544136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1394/2856] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:19:17 2022] DELETE /identity/v3/projects/7a53502e395a4c96879bcdfbbe79d727 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:17.550579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.551104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.551104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.588752 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:17.589968 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:17.648857 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1463/2857] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:17.658602 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.658901 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.659182 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.679340 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:17.680579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:17.688421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1395/2858] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:17.692852 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.702530 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd57b82f-9acf-4d41-840c-1543e3b8f106 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.707597 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.708232 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.708514 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.708782 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] PATH_INFO: `/v3/domains` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.709423 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81466) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:19:17.709727 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.711341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.723407 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1464/2859] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:19:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:19:17.727597 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.736817 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4525a-bbbe-4bba-81f9-7b4787c1dfe2 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.742009 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.742604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.742883 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.743151 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] PATH_INFO: `/v3/projects` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.743816 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.745724 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.772772 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1396/2860] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:19:17 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:17.776799 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.787292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48465059-c6e2-495b-8148-24467c21e19a admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.844294 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.844899 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.845178 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.845450 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] PATH_INFO: `/v3/users` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.846110 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.848864 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.870104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1465/2861] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:19:17 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:19:17.874440 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.885068 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f3a86-ec9e-4131-aec4-14bbddcc0559 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.889589 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.890445 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.890721 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.890985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.891608 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.893222 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.904401 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1397/2862] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:17.907560 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.918337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff616e-b953-4dd1-a20c-2ce44ecee370 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.923067 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.923664 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.924091 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.924326 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.925139 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.926750 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b61cd-6304-4072-80ed-3e72750b5878 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.932048 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:17.939586 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1466/2863] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:19:17.943775 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:17.954003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da6316-f5b7-4f78-9dd4-ccabe571214f admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:17.959559 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:17.960241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:17.960521 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:17.960786 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] PATH_INFO: `/v3/projects/765a8b801ae84d9f9173d573c35fc168/users/09a8bdade06343728b7caf50c628121b/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:17.976435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=765a8b801ae84d9f9173d573c35fc168, user_id=09a8bdade06343728b7caf50c628121b, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:17.979066 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:17.991712 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1398/2864] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:19:17 2022] PUT /identity/v3/projects/765a8b801ae84d9f9173d573c35fc168/users/09a8bdade06343728b7caf50c628121b/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:19:18.003954 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:18.004374 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:18.004771 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:18.032583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] MFA Rules not processed for user `09a8bdade06343728b7caf50c628121b`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:19:18.033733 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:18.099525 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1467/2865] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:19:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:19:18.111470 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:18.113951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:18.170149 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02462515-0e95-4a02-976e-e8fac3a61d44 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:18.173718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:18.174319 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:18.174593 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:18.175514 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:18.183716 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:18.185172 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:18.193935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1399/2866] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:18 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:23.390945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:19:23.392901 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:23.464254 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f215ff-3d6e-403d-8c3f-d38afda95c0c None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:19:23.469337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:19:23.469962 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:19:23.470241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:19:23.471213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:19:23.473085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:19:23.485306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:19:23.486982 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7d30b-3be1-491e-ae82-6fd689eee4f4 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:19:23.496685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1468/2867] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:19:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:19:28.189962 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:19:33.492498 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=64 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:04.306792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:04.372805 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054afa0d-568c-441c-9560-888f4cfe6f66 service glance] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:04.377368 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:04.377994 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:04.378273 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:04.378548 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] PATH_INFO: `/v3/users/09a8bdade06343728b7caf50c628121b` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:04.379663 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09a8bdade06343728b7caf50c628121b)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:04.381626 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:04.428279 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09a8bdade06343728b7caf50c628121b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:04.428972 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09a8bdade06343728b7caf50c628121b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:04.441569 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09a8bdade06343728b7caf50c628121b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:04.449164 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09a8bdade06343728b7caf50c628121b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:04.457092 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09a8bdade06343728b7caf50c628121b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:04.463083 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1400/2868] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:20:04 2022] DELETE /identity/v3/users/09a8bdade06343728b7caf50c628121b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:04.468348 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:04.468847 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:04.468960 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:04.487435 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:04.488668 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:04.547195 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1469/2869] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:04.556669 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:04.558046 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:04.617006 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b876702-7328-4186-ad0d-da48757dd311 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:04.620677 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:04.621296 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:04.621570 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:04.621842 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:04.631316 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:04.633073 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027b03-12c4-415d-ab87-a5ea09d7a969 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:04.643034 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1401/2870] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:04.922898 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:04.925292 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:04.938341 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba23d-2b03-4500-b16b-5d6d057387b7 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:04.942014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:04.942659 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:04.942944 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:04.943217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] PATH_INFO: `/v3/projects/765a8b801ae84d9f9173d573c35fc168` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:04.944398 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] RBAC: Authorizing `identity:delete_project(project_id=765a8b801ae84d9f9173d573c35fc168)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:04.946336 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.002452 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765a8b801ae84d9f9173d573c35fc168', 'request_id': 'req-ad530386-7c03-4a9e-8767-81053c6d163b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:05.010396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765a8b801ae84d9f9173d573c35fc168', 'request_id': 'req-ad530386-7c03-4a9e-8767-81053c6d163b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:05.019515 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765a8b801ae84d9f9173d573c35fc168', 'request_id': 'req-ad530386-7c03-4a9e-8767-81053c6d163b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:05.030401 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] The token cache is being invalidate because project 765a8b801ae84d9f9173d573c35fc168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:20:05.030689 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:05.031346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-ad530386-7c03-4a9e-8767-81053c6d163b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:05.032855 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1470/2871] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:20:04 2022] DELETE /identity/v3/projects/765a8b801ae84d9f9173d573c35fc168 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:05.038367 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.038654 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.038927 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.061119 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:05.062422 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.083509 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:05.136065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1402/2872] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:05.145851 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.146315 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.146698 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.169601 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:05.171641 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.181323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1471/2873] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:05.186148 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.197158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a42a2-3de0-48c7-960c-6083fb2c6cbf None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.203671 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.204412 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.204697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.204973 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.205732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:20:05.206284 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.208391 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.222528 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1403/2874] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 10:20:05.226832 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.238443 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5a111-67fa-4c47-8e44-6341da83fefd None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.243684 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.244405 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.244685 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.244958 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.245708 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.247638 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.281500 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1472/2875] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:05.285832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.297944 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d29b08-54d7-4ef2-804d-386888fe878d admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.360708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.362012 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.362369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.362718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.363604 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.365718 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.392329 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1404/2876] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 10:20:05.396583 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.408396 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c43c7-17fa-4fe6-894d-c98c0e846566 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.413494 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.414187 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.414527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.414863 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.415577 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.417574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.431127 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1473/2877] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 10:20:05.434158 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.447057 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db941d20-4015-4ec3-b662-ebada8d0d71e admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.454135 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.455345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.455728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.456133 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.456897 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.458839 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a0a65-d620-4624-ba8c-fc1c4ec6cd8d admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.473827 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1405/2878] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:05.477887 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.489523 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2dd708-065f-4b78-b4ca-7c6f0532c2ca admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.494456 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.495163 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.495434 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.495702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] PATH_INFO: `/v3/projects/0659dc74da52481f834f40b775a294c2/users/dffbd23e69714532a16dc1cef9ef826a/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.513531 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0659dc74da52481f834f40b775a294c2, user_id=dffbd23e69714532a16dc1cef9ef826a, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.516471 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.532519 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1474/2879] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:05 2022] PUT /identity/v3/projects/0659dc74da52481f834f40b775a294c2/users/dffbd23e69714532a16dc1cef9ef826a/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:05.544154 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.544461 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.544736 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.579413 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] MFA Rules not processed for user `dffbd23e69714532a16dc1cef9ef826a`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:05.581061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.660003 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1406/2880] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:05.672849 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:05.674629 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:05.698702 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:05.743420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149b7de-85af-4b5d-8cb8-615f079d1813 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:05.747480 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3061244943c84217bf9ba30224cd4951', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:05.748140 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:05.748419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:05.748790 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:05.757994 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:05.759430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373c4113-f07a-43e9-b895-3076b0003fa1 service glance] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:05.769547 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1475/2881] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:15.654321 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:15.765337 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=17 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:37.772265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:37.834908 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bf5e64-c302-437b-aaac-fcaf609cb1bf None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:37.839574 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:37.840234 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:37.840511 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:37.840779 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] PATH_INFO: `/v3/users/dffbd23e69714532a16dc1cef9ef826a` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:37.841895 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dffbd23e69714532a16dc1cef9ef826a)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:37.843832 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:37.891363 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dffbd23e69714532a16dc1cef9ef826a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:37.892049 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dffbd23e69714532a16dc1cef9ef826a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:37.904085 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dffbd23e69714532a16dc1cef9ef826a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:37.911348 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dffbd23e69714532a16dc1cef9ef826a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:37.918345 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dffbd23e69714532a16dc1cef9ef826a'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:37.924232 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1407/2882] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:20:37 2022] DELETE /identity/v3/users/dffbd23e69714532a16dc1cef9ef826a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:37.928746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:37.929225 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:37.929346 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:37.948940 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:37.950150 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:38.009343 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1476/2883] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 10:20:38.019134 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.020598 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:38.078417 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29207723-665c-4c6f-a343-b7034098f439 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.082104 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.082708 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.082986 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.083257 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.092272 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.093694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347eabb1-1401-46e9-b692-b0b2ba4e6b2d service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.102984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1408/2884] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:38.371455 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.373103 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:38.387018 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0fc865-3ac6-4afa-8396-598350004fe0 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.391615 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.392935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.393212 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.393650 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] PATH_INFO: `/v3/projects/0659dc74da52481f834f40b775a294c2` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.394794 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0659dc74da52481f834f40b775a294c2)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.397122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.445038 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0659dc74da52481f834f40b775a294c2', 'request_id': 'req-0a1bd495-3ab4-4341-9825-4d0ba9589241'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:38.455243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0659dc74da52481f834f40b775a294c2', 'request_id': 'req-0a1bd495-3ab4-4341-9825-4d0ba9589241'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:38.462430 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0659dc74da52481f834f40b775a294c2', 'request_id': 'req-0a1bd495-3ab4-4341-9825-4d0ba9589241'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:38.472716 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] The token cache is being invalidate because project 0659dc74da52481f834f40b775a294c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:20:38.473001 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:38.473657 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-0a1bd495-3ab4-4341-9825-4d0ba9589241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:38.475224 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1477/2885] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:20:38 2022] DELETE /identity/v3/projects/0659dc74da52481f834f40b775a294c2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 10:20:38.480087 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.480380 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.480646 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.501719 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:38.503193 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:38.520869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:38.564732 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1409/2886] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:38.574412 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.574707 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.574985 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.594459 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:38.595681 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:38.603745 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1478/2887] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:38.608035 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.618243 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256eb87-cce1-4444-9d4d-01c47a7e2673 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.622951 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.623551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.623869 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.624152 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] PATH_INFO: `/v3/domains` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.624797 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81467) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 10:20:38.625102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.626697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.639067 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1410/2888] 10.222.0.78 () {58 vars in 1153 bytes} [Wed Aug 24 10:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 10:20:38.643356 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.652966 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d044d4b-a0aa-4d88-a09b-dac6af5ae6ec None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.657647 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.658249 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.658604 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.658806 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] PATH_INFO: `/v3/projects` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.659432 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.661366 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.700072 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1479/2889] 10.222.0.78 () {60 vars in 1153 bytes} [Wed Aug 24 10:20:38 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:38.704603 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.716080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37ce482-cd90-40c3-9638-f9ab61f69738 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.775004 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.775709 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.775905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.776163 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] PATH_INFO: `/v3/users` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.776792 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.778715 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.801385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1411/2890] 10.222.0.78 () {60 vars in 1144 bytes} [Wed Aug 24 10:20:38 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 10:20:38.806188 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.816576 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb941-7990-4c94-a3ab-e72678c3a37c admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.822362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.822957 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.823236 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.823507 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.824192 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.825783 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.835585 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1480/2891] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:38.840405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.850757 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e2ea16-ba71-4f4f-9005-1b813b132894 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.855957 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.856544 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.856823 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.857090 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.857836 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.859403 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf4ea-8320-48d6-b2e3-a47d55282b1e admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.873165 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1412/2892] 10.222.0.78 () {58 vars in 1122 bytes} [Wed Aug 24 10:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 10:20:38.877420 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:38.887920 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84750d52-d2ae-4136-a560-96f385110b34 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:38.892512 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:38.893157 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.893445 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.893718 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] PATH_INFO: `/v3/projects/e0f4add9ac894690990b023145ae9f1a/users/4910a16422b249439f6c2eb5ed657386/roles/4156f70d4c1744029972dc81f38353a8` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.908402 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f4add9ac894690990b023145ae9f1a, user_id=4910a16422b249439f6c2eb5ed657386, role_id=4156f70d4c1744029972dc81f38353a8)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:38.911060 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:38.924361 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1481/2893] 10.222.0.78 () {60 vars in 1483 bytes} [Wed Aug 24 10:20:38 2022] PUT /identity/v3/projects/e0f4add9ac894690990b023145ae9f1a/users/4910a16422b249439f6c2eb5ed657386/roles/4156f70d4c1744029972dc81f38353a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:38.935136 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:38.935427 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:38.935703 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:38.963743 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] MFA Rules not processed for user `4910a16422b249439f6c2eb5ed657386`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:38.964988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:39.029188 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1413/2894] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:39.040883 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:39.042645 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:39.056811 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:39.126233 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d37c3-13a3-4294-b9c0-e640f312cb10 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:39.130527 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:39.131362 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:39.131845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:39.132321 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:39.141564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:39.142996 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0fd3b1-c080-4952-a4d7-6fe8bfeca9f1 service cinder] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:39.152882 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1482/2895] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:39.177565 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:39.197230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:39.243176 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77785b3b-7fbc-4773-86ae-5ae94f404324 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:39.247773 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:39.248421 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:39.248723 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:39.249001 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] PATH_INFO: `/v3/users/4910a16422b249439f6c2eb5ed657386` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:39.250132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4910a16422b249439f6c2eb5ed657386)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:39.252376 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:39.297936 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4910a16422b249439f6c2eb5ed657386'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.298591 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4910a16422b249439f6c2eb5ed657386'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.310228 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4910a16422b249439f6c2eb5ed657386'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.317984 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4910a16422b249439f6c2eb5ed657386'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.325130 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4910a16422b249439f6c2eb5ed657386'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.330884 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1414/2896] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:20:39 2022] DELETE /identity/v3/users/4910a16422b249439f6c2eb5ed657386 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:39.335419 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:39.335706 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:39.336504 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:39.355592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] MFA Rules not processed for user `0f68be8cf7b24b869bf665ebc09aa5e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:20:39.356746 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:39.413989 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1483/2897] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:20:39.423696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:39.425127 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:39.484328 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422a21d-ca5c-4d1c-bf9f-5dcb43fa95e9 admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:39.487940 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:39.488547 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:39.488826 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:39.489099 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:39.498769 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:39.500247 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:39.510425 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1415/2898] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:39.787741 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:39.789197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:39.801574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d44936-d931-4255-98f3-0de32b2b7c7f None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:39.807099 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:39.807769 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:39.808066 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:39.808347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] PATH_INFO: `/v3/projects/e0f4add9ac894690990b023145ae9f1a` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:39.809473 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f4add9ac894690990b023145ae9f1a)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:39.811394 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:39.860564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f4add9ac894690990b023145ae9f1a', 'request_id': 'req-5ed02f25-87aa-441c-a9f8-7b4661b0f310'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.868446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f4add9ac894690990b023145ae9f1a', 'request_id': 'req-5ed02f25-87aa-441c-a9f8-7b4661b0f310'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.874808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f4add9ac894690990b023145ae9f1a', 'request_id': 'req-5ed02f25-87aa-441c-a9f8-7b4661b0f310'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.886040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] The token cache is being invalidate because project e0f4add9ac894690990b023145ae9f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81466) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:20:39.886322 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.886947 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:20:39.888460 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1484/2899] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:20:39 2022] DELETE /identity/v3/projects/e0f4add9ac894690990b023145ae9f1a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:20:49.505745 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:49.882933 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:20:57.215741 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:57.217245 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:57.229697 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecce2-9a4d-4f23-8adf-7669f11e24ab service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:57.279435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:57.280079 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:57.280360 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:57.280636 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:57.282404 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:57.351988 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:57.353369 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe19d3f-4430-4028-981d-dcbd1e51d264 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:57.364444 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1416/2900] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:20:58.553661 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:20:58.555014 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:58.567168 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed02f25-87aa-441c-a9f8-7b4661b0f310 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:20:58.571397 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:20:58.572044 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:20:58.572323 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:20:58.572596 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:20:58.574286 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:20:58.588606 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:20:58.590039 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:20:58.601040 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1485/2901] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:20:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:00.893780 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] REQUEST_METHOD: `POST` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:00.894285 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:00.894385 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:00.914143 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.auth.core [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] MFA Rules not processed for user `2e137db209634ae4b760eba456203e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81467) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:21:00.915351 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:00.979867 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1417/2902] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:21:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:21:07.677592 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:21:07.679123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:07.689753 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4700-9e1f-4db9-a5f9-ac21726ef32c service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:21:07.693476 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:21:07.694085 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:21:07.694365 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:21:07.694637 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:21:07.696403 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:21:07.706347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:21:07.708572 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c76a7-4f37-481b-8c06-5514bec5dc06 service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:21:07.717697 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1486/2903] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:21:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:21:10.972531 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=79 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:21:17.713552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=12 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:26.647291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:26.648799 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:26.705843 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9bd2be-4883-4cf8-81c8-1fb9ae4b2230 None None] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:26.709497 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b171220975d48488be864348cc9aca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:26.710241 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:26.710520 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:26.710793 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:26.712551 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:26.723105 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:26.724575 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:26.734021 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1418/2904] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:36.730100 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=26 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:38.806632 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] REQUEST_METHOD: `POST` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:38.807160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:38.807160 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:38.827104 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.auth.core [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] MFA Rules not processed for user `2e137db209634ae4b760eba456203e66`. Rule list: `[]` (Enabled: `True`). {{(pid=81466) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 10:23:38.828344 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:38.835555 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1487/2905] 10.222.0.78 () {58 vars in 958 bytes} [Wed Aug 24 10:23:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 10:23:39.153829 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:39.163312 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbd272-fcd2-4483-be79-be1afd0ce3cd service cinder] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:39.167561 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:39.168197 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:39.168474 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:39.168755 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:39.170475 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:39.180112 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:39.181676 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:39.189985 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1419/2906] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:40.660687 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:40.662198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.733945 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159f563-249d-4956-a2d8-c72e164ff199 None None] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:40.739217 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:40.739845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:40.740129 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:40.740404 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:40.742643 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:40.753861 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:40.755431 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:40.766264 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1488/2907] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:43.381643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:43.383664 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.403126 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948cf54-f6d4-4ff6-b12f-a4276db30a72 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:43.409579 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:43.410265 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:43.410614 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:43.410965 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:43.414220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:43.430394 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:43.432269 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:43.444054 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1420/2908] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:23:50.762037 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=35 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:53.438970 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=12 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:23:56.490608 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:56.504669 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7dfe64-138d-41be-9fec-c9118590f60d service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:56.511135 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:56.511777 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:56.512094 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:56.512371 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:56.514197 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:56.527312 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:56.528939 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:56.540372 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1489/2909] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:23:59.635111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:23:59.644478 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6e0891-2d5a-4b85-a3a9-ffe80930b6a4 service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:23:59.648072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:23:59.648661 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:23:59.648935 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:23:59.649199 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:23:59.650896 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:23:59.713643 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:23:59.715124 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:23:59.724748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1421/2910] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:23:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:06.535056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:09.720530 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=30 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:24:28.349241 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:24:28.361410 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc1192-583c-4f0c-9125-e92ecad26b69 service nova] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:24:28.366425 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:24:28.367024 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:24:28.367306 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:24:28.367579 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:24:28.369992 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:24:28.381446 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:24:28.383029 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:24:28.393183 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1490/2911] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:24:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:24:38.388808 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=6 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:01.809326 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:01.810853 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:01.829873 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f1fbd9-70f1-45db-a133-8d763ab394a3 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:01.836210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7f645f811e804835a793f372a1ac7111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:01.837215 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:01.837694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:01.838179 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:01.840230 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:01.855696 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:01.857687 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:01.870693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1422/2912] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:01.989815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:02.000012 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43a01d-ae1a-4c46-9947-47100e2655fa service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:02.003845 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:02.005000 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:02.005281 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:02.005552 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:02.018770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:02.020291 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:02.032710 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1491/2913] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 10:26:03.228159 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:03.237847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9432-0451-4439-901d-6b6081541e5f service nova] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:03.241825 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:03.242442 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:03.242728 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:03.243005 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:03.254725 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:03.256210 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:03.268352 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1423/2914] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:08.766946 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:08.777495 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d37b-fa26-4f47-95d9-3b972049c24b service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:08.781470 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:08.782082 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:08.782363 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:08.782631 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:08.792017 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:08.794484 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:08.804931 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1492/2915] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:13.263909 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=12 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:14.434102 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:14.443128 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ce5f5-50fb-42c4-8553-ca567e2a906c service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:14.447783 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77e0d9362b3241d3b888e1cb0a2b5f4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:14.448503 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:14.449061 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:14.449289 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:14.451038 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:14.512637 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:14.514080 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:14.525694 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1424/2916] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:17.348683 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:17.362930 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3290def-6ec2-49fb-bcaa-8512f016084c service neutron] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:17.369209 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:17.370368 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:17.370829 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:17.371268 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:17.373331 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:17.393016 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:17.394770 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:17.412347 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1493/2917] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:20.246645 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:20.262859 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442bc81-b444-476c-ba55-312aa7d95b41 service neutron] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:20.268132 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:20.268735 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] REQUEST_METHOD: `GET` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:20.269016 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:20.269291 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:20.271072 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:20.284905 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:20.286662 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:20.298416 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1425/2918] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:26:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:26:20.312564 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:20.314742 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:20.330772 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086b615-4a89-4b6f-9c53-c987c60da2c8 service placement] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:20.338122 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:20.339162 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:20.339605 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:20.340144 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] PATH_INFO: `/v3/users/2e137db209634ae4b760eba456203e66` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:20.341824 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e137db209634ae4b760eba456203e66)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:20.345056 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:20.394891 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e137db209634ae4b760eba456203e66'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.395551 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e137db209634ae4b760eba456203e66'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.403757 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e137db209634ae4b760eba456203e66'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.415835 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e137db209634ae4b760eba456203e66'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.422804 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e137db209634ae4b760eba456203e66'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.428900 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1494/2919] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:20 2022] DELETE /identity/v3/users/2e137db209634ae4b760eba456203e66 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:20.620111 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:20.631030 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0148-1be6-4579-af03-fe6df30faa9f service placement] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:20.690274 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:20.690894 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:20.691171 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:20.691441 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] PATH_INFO: `/v3/projects/34855b10c042463d8c5067cef47b956d` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:20.692634 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] RBAC: Authorizing `identity:delete_project(project_id=34855b10c042463d8c5067cef47b956d)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:20.694537 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:20.741435 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34855b10c042463d8c5067cef47b956d', 'request_id': 'req-dcec0076-e903-45b0-8508-3c7d9f0e672b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.749437 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34855b10c042463d8c5067cef47b956d', 'request_id': 'req-dcec0076-e903-45b0-8508-3c7d9f0e672b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.755758 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34855b10c042463d8c5067cef47b956d', 'request_id': 'req-dcec0076-e903-45b0-8508-3c7d9f0e672b'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.766373 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] The token cache is being invalidate because project 34855b10c042463d8c5067cef47b956d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:20.766693 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.767332 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.768847 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1426/2920] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:20 2022] DELETE /identity/v3/projects/34855b10c042463d8c5067cef47b956d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:20.772815 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:20.774724 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:20.787594 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17756c32-50ae-412f-8e45-81fc98d096e4 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:20.845765 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:20.846389 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:20.846666 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:20.846935 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] PATH_INFO: `/v3/users/7caf67dcec664fe8be210ead7e51f54b` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:20.848213 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7caf67dcec664fe8be210ead7e51f54b)` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:20.850121 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:20.889164 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:20.900359 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7caf67dcec664fe8be210ead7e51f54b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.900942 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7caf67dcec664fe8be210ead7e51f54b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.907619 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7caf67dcec664fe8be210ead7e51f54b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.920257 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7caf67dcec664fe8be210ead7e51f54b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.927205 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.notifications [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7caf67dcec664fe8be210ead7e51f54b'} {{(pid=81466) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:20.933820 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1495/2921] 10.222.0.78 () {58 vars in 1224 bytes} [Wed Aug 24 10:26:20 2022] DELETE /identity/v3/users/7caf67dcec664fe8be210ead7e51f54b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:21.108399 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Authenticating user token {{(pid=81467) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:26:21.110142 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.fernet_utils [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81467) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:26:21.123229 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec0076-e903-45b0-8508-3c7d9f0e672b admin admin] Validating token access rules against request {{(pid=81467) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:26:21.132307 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:21.186921 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f68be8cf7b24b869bf665ebc09aa5e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7790bbe18b09475b81076e89944d2568', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81467) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:26:21.187468 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:26:21.187748 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:26:21.188056 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] PATH_INFO: `/v3/projects/3d8ac6f22ebf448681daf161ace21b65` {{(pid=81467) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:26:21.189168 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d8ac6f22ebf448681daf161ace21b65)` {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:26:21.191065 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] RBAC: Authorization granted {{(pid=81467) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:26:21.236856 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d8ac6f22ebf448681daf161ace21b65', 'request_id': 'req-306bf0ff-6086-486d-8377-8eb4794aec78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.243932 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d8ac6f22ebf448681daf161ace21b65', 'request_id': 'req-306bf0ff-6086-486d-8377-8eb4794aec78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.250063 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d8ac6f22ebf448681daf161ace21b65', 'request_id': 'req-306bf0ff-6086-486d-8377-8eb4794aec78'} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.260298 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] The token cache is being invalidate because project 3d8ac6f22ebf448681daf161ace21b65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81467) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 10:26:21.260585 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.261220 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG keystone.notifications [None req-306bf0ff-6086-486d-8377-8eb4794aec78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81467) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 10:26:21.262698 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: [pid: 81467|app: 0|req: 1427/2922] 10.222.0.78 () {58 vars in 1233 bytes} [Wed Aug 24 10:26:21 2022] DELETE /identity/v3/projects/3d8ac6f22ebf448681daf161ace21b65 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 10:26:30.930239 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:26:31.257405 dv-cinder-854269-2-iscsi devstack@keystone.service[81467]: DEBUG dbcounter [-] [81467] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81467) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 10:29:34.318704 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Authenticating user token {{(pid=81466) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 10:29:34.320198 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:34.461424 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25139540-8a98-422c-bcd3-ff774313afd1 admin admin] Validating token access rules against request {{(pid=81466) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 10:29:34.520964 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b0a88b1bdce5483c95b59921dbfc5660', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35e2575a816145b297430e41f586390d', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81466) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 10:29:34.521574 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] REQUEST_METHOD: `GET` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 10:29:34.521853 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] SCRIPT_NAME: `/identity` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 10:29:34.522123 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81466) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 10:29:34.523895 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.fernet_utils [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81466) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 10:29:34.535287 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 10:29:34.536842 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82fe4b6-ab10-4983-a175-4015ab910638 service placement] RBAC: Authorization granted {{(pid=81466) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 10:29:34.547918 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: [pid: 81466|app: 0|req: 1496/2923] 10.222.0.78 () {60 vars in 1365 bytes} [Wed Aug 24 10:29:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 10:29:44.544243 dv-cinder-854269-2-iscsi devstack@keystone.service[81466]: DEBUG dbcounter [-] [81466] Writing DB stats keystone:SELECT=89 {{(pid=81466) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}